THE AIMS OF THE PROJECT

In his 1965 book, Understanding Media, Marshall McLuhan wrote, "The hybrid or meeting of two media is a moment of truth and revelation of which new form is born...The crossings of media release great force." In many ways, this sort of bringing together is exactly what researchers have done with virtual reality. The technologies of computer graphics, data communications, and computer programming have been loosely melded with the technologies of the telephone, the television, and the video game. The result is a single, ever-changing technology, immensely superior to each of its predecessors.

Although the technology is mature enough to have different applications, there are key issues to be resolved for its use for medical applications.

• The product seem to be "a solution in search of a problem". As with early computer graphics products, the entry-level costs are relatively prohibitive. A complete VR environment, including workstations, goggles, body suits, and software, is in the range of KEcu 70.000 to KEcu 1.000.000.
• The hyperbole and sensational press coverage associated with some of this technology has led many potential users to overestimate the actual capabilities of existing systems. Many of them must actually develop the technology significantly for their specific tasks. Unless their expertise includes knowledge of the human-machine interface requirements for their application, their resulting product will rarely get beyond a "conceptual demo" that lacks practical utility.
• The premise of VE seems to be to enhance the interaction between people and their systems. It thus becomes very important to understand how people perceive and interpret events in their environments, both in and out of virtual representation of reality. We must address issues of human performance to understand how to develop and implement VE technology that people can use comfortably and effectively. Fundamental question remain about how people interact with the systems, how they may be used to enhance and augment cognitive performance in such environments, and how they can best be employed for instruction, training, assessment, rehabilitation and other people oriented applications.

1. Objectives of the project

2. Key issues of the VREPAR project

3. A framework for future researches in VR human issues

1. Objectives of the project

It is precisely one of the goals of the project to overcome those shortcomings in order to develop a demonstrator of a virtual environment that can be effectively used by end users. The main goal of the project is to build a demonstrator of a virtual reality system (Virtual Reality All-purpose Modular System - VRAMS), based on a modular architecture, to be used for psychoneurophysiological assessment and rehabilitation.

Various studies aimed at the development of virtual reality systems for this type of application have been under way for some time now, but the existing systems have a series of problems that limit their real possibilities of use:

• Cost: Although some attempts have been made to use PC-based virtual reality systems, the majority of the existing systems use RISC platforms whose cost is beyond the means of a normal Hospital Centre or Department.
• Lack of reference standards: Almost all of the applications in this sector can be considered "one-off" creations tied to their development hardware and software, which have been adjusted in the field by a process of trial and error. This makes them difficult to use in contexts other than those in which they were developed.
• Non-interoperability of the systems: Although it is theoretically possible to use a single virtual reality system in many different applications, none of the existing systems can be easily adapted to different tasks. This means that two different wards/departments within the same organisation may find themselves having to use two different VR systems because of the impossibility of adapting one single system to their different needs.
• Lack of network connections: The possibility of establishing network connections could considerably increase the ways of using the systems by introducing their possible remote use but, with the exception of a few prototypes, both hardware and software problems make it difficult to network the existing systems.

With the development of VRAMS, VREPAR has the following objectives:

a) To develop a virtual reality system for the medical market that can be marketed at a price which is accessible to its possible end-users (hospitals, universities and research centres) and which would have the modular, connectability and interoperability characteristics that the existing systems lack;

b) To develop three hardware/software modules for the application of VRAMS in psychoneurophysiological assessment and rehabilitation. The chosen development areas are eating disorders (bulimia, anorexia and obesity), movement disorders (Parkinson's disease and torsion dystonia) and stroke disorders (unilateral neglect and hemiparesis).

c) To define reference standards and parameters relating to technological and experimental factors, which can also be used by third parties. In particular, VREPAR aims at:

- defining a standard for software development and a series of hardware specifications to be used in the development of further VRAMS modules;

- identifying the factors affecting individual experiences in a virtual environment;

2. Key issues for the VREPAR project

Undoubtedly the construction of different VR modules brings out many interesting and challenging design questions. The following key points have emerged from our survey of PC related market that are relevant for VREPAR Project's aims:

• PC hardware market is chaotic and restless, thus compelling not to bet on particular architectures but only on converging tools while waiting for wider standardisation;
• Virtual Reality building software packages differ in terms of hardware support and of implemented standard formats;
• Each market segment has its peculiar solutions and "best buys", hardly replaceable by different choices;
• A convergence is necessary between these different areas in order to achieve a sort of "work-in-progress" standard, based upon the common support of interfacing tools;
• VRML language and world building/browsing tools do provide such chance of convergence.
  

A suitable approach is in considering these points not as constraints for our research but to turn them into advantages for the successful outcome of the project. The instability of the market can drive us towards the adoption of a standard highly inter-operable and multi-platform like VRML. It may be widely diffusable because almost all VR packages support the possibility of importing/converting VRML worlds. In terms of low cost technology it satisfies the requirements under the most convincing circumstances because a VRML application is unbounded from platform specifications and available to immediate exploration in the Internet which we can consider under this perspective the most promising method for low-cost VR. The Internet exploitation opens a wide set of possibility linked to the envisaging of medical practices as activities freed from time and location constraints. VR could assist highly demanded operations such as tele-diagnostics, tele-therapy and many others both as one-to one or as collaborative processes among different roles and actors opening exciting new perspectives for medical VR.

But the problem of VREPAR is not just a technical problem of assembling highly technical devices, it is also a problem of rendering the efficacy and successful outcome of the project with the intrinsic guarantee that any VR interaction treatment whether be diagnostic or therapeutic must definitely be not harmful and cause any danger to patients and users. Like for any other medical aim a specific concern on the absence of collateral effects and accurate testing on the result of somministration is mandatory due to the high responsibility of dealing primarily with human health. There is a literature on the effects that VR interaction may cause to users, these effects while being limited to special sectors of population can appear and undermine the clinical benefit with unpredictable and unwanted results. So in this perspective it has to be considered one of the key issues an accurate and attentive consideration of human factors involved, this has to be achieved at this stage of the project were hardware software characteristics must be able to set their use to the mandatory concern of human health and safety.

Professional literature on Human Computer Interaction - HCI specifically requires from the prototyping phase special consideration of these issues which are not ignored in the project and will be attentively treated as the work will progress.

Another consideration can be made for what concerns the more accurate deepening and tuning of the different modules to the specific cognitive and social contexts of use.

3. A framework for future researches in VR human issues

As underlined before, the problem of VREPAR is not just a technical problem of assembling highly technical devices. A main aim of the project is to assure that VR interaction treatment, whether be diagnostic or therapeutic, is not harmful and cause any danger to patients and users. To establish a framework for the future research in VR human issues, a method is needed to quantify the temporal performance of virtual reality systems, ensuring the HCS characteristics can cope with them since the prototyping phase. Infact, as we seen in the human factor part, many of the undesirable effects of exposures to virtual reality environments can be ascribed to limitations in the temporal performance of the system hardware. These limitations result in lags between movements of the head or hands and corresponding movements of displayed images.

A mathematical model is described below which could provide the basis for such a framework. The inputs to the model are either the angular displacements of tracked objects, or recorded head motions (Lewis and Griffin, 1996).

3.1 Input output models of the temporal performance of the VR systems

3.2 Applications of systems models

3.3 Recommendation for the design of applications

3.1 Input-output models of the temporal performance of virtual reality systems

Transfer function models are used to represent the sampling delays in a head position sensor, image processor and display. Transfer function models can also be used to represent the tracking behaviour of users and their vestibular and ocular responses to head and image motions (Lewis and Griffin, 1996). The dynamic of the various components of the model are discussed in below.

The transfer functions of the system components can be combined into the input-output model shown in Figure 2. The primary inputs to the model are the displacements of the visual target. A head motion time history may also be input directly to the model, in which case the elements of the human operator tracking model, H_h(s) and R(f) would be set to zero.

3.1.1 Modelling the temporal characteristics of the human operator

The head tracking response

The human operator in a continuous tracking task can be modelled as a linear transfer function, H_h(f), where:

where O_h(f) and I(f) are frequency domain representations of the head position output, o_h(t), and the target position input, i(t). From an engineering point of view a human operator is a non-linear control element, but Krendel and McRuer (1965) showed that, with particular inputs, subjects responded similarly to equivalent linear systems. However, to completely describe the system output it was necessary to include a "remnant" component, r(t). The remnant comprises a relatively small signal which is added to the output of the linear model to account for differences between the response of the operator and the time-invariant linear relationship between the system input and both the tracking error and the system output. This type of model is referred to as a quasi-linear model of the human operator. The remnant is modelled as normally distributed broad-band random noise.

So and Griffin (1995a) have measured closed-loop transfer functions for head-aiming. A cross-hair aiming reticle was displayed in the centre of a monocular helmet mounted display. A circular target was driven by independent random functions in both the pitch and the yaw axes. The head coupled system had an inherent lag of approximately 40 ms between head movements and the corresponding movement on the display (comprising the lag in the head position sensor, the computation time and the update rate of the display). An additional time delay, which was varied between 0 ms and 160 ms, was imposed by a computer.

Figure 3 shows mean head tracking transfer functions in the pitch and yaw axes measured with five values of imposed system lag. At frequencies below 0.4 Hz the modulus (i.e. the gain) of the human operator transfer functions were all close to unity but at higher frequencies the gain increased with increasing imposed display lag. The increased gain at higher frequencies was believed to be a strategy used by the subjects to compensate for the increased lag in the system. The phase lags reflect delays in the operator's response, relative to the target motion. The phase lags at around 0.1 Hz were shown to decrease significantly with increasing lag, but the reduction was not sufficient to compensate for the imposed display lag. The phase response at 0.1 Hz was important since a large proportion of the energy in the target motion was around this frequency. Although there was a decrease in the phase lag of the subjects with increasing system lag at low frequencies, the phase lags above 0.5 Hz increased with increasing system lag. This is a typical response for a system consisting of a lead-lag filter and a lag: lead generation at low frequencies is accompanied by increases in response lag at higher frequencies, and a consequent reduction in tracking bandwidth (McRuer, 1973).

The vestibulo-ocular reflex

The vestibulo-ocular reflex (VOR) induces movements of the eyes which compensate for movements of the head. The VOR stabilises the eyes in space during body motion. This makes it possible to view objects which are fixed in space without smearing of the retinal image. The frequency response characteristics of the compensatory eye movements induced by angular oscillation in yaw have been measured by, for example, Benson and Barnes (1978). Barnes (1980) has modelled the slow phase (compensatory) eye movements evoked by yaw axis rotation by the following transfer function:

where s is the complex radial frequency; is the angular displacement of the eye; is the angular displacement of the head; T=0.005 s; T_A=0.2 s; T_B=15 s; T_C= 0.125 s; T_D= 0.002 s; T_AD=80 s and K_C=0.7.

The pursuit reflex

The VOR can be overridden at low frequencies by a pursuit tracking reflex. Benson and Barnes (1978) have discussed methods for modelling this visually-driven response. The pursuit reflex has been shown to break down when the velocity of a viewed object is greater than 40 to 60 degrees per second or the frequency of the movement is above about 1 Hz (Benson and Barnes, 1978).

3.1.2 Modelling the temporal characteristics of system components

Head position sensor

Typical head pointing systems sample the angular displacement of the head with sample rates between 30 and 60 samples per second. The output signal of the head pointing system, (s), in response to head displacement, (s), can be represented by:

where *(s) is the signal (s) sampled at discrete intervals of T seconds and is given by:

If the system operates at a sampling rate of 30 samples per second, T= 2/_s = 0.033 s. H_delay(s) represents a time delay of Tseconds and is equivalent to:

H_hold(s) is a zero-order hold and is equivalent to:

Image processing

The image processor is a computer which takes the head angle measured by the head pointing system and renders the images which are displayed on the head-mounted display at appropriate locations in the video frame. The computation will impose a time delay and a sample-and-hold effect. Hence the transfer function, H_c(s), has the same form as that of the head pointing system. The computation time, T, is the inverse of the system frame rate.

Head-mounted display

The response and frame rate of the display combine to produce a sample-and-hold effect, hence the transfer function representing the effect of the display on the position of a moving image will be approximated by:

with a frame rate of 60 frames per second T= 2/_s = 0.0167 s. H_hold(s) is a zero-order hold given by:

Predictive filter

The characteristics and implementation of phase-lead filters for predicting future head motions have been discussed by So (1995) and So and Griffin (1996). Alternative methods are cited by Wioka (1995). Alternative head position prediction algorithms may be implemented in the model to determine the extent to which they reduce the effects of system lags.

3.2 Application of system models

Evaluation of visual-vestibular interaction

Perception of rotational self-motion may be examined using models such as that proposed by Zacharias and Young (1981). The inputs to this model are the head velocities sensed by the semi-circular canals, _ves, and by peripheral vision (i.e. the world-stabilised background image), _vis. the extent of potential cue conflicts between visual and vestibular perception of self-motion may be derived from the difference between the two signals.

Evaluation of visual-vestibular interaction in the perception of angular self-motion.

This figure shows how that generic model may be adapted to evaluate potential cue conflicts. The figure also shows the velocity difference between the head velocity and image velocity time histories. In this example the system update rate was 10 Hz and the lag in the head pointing system was 15 ms. A conflict index was derived from the relative magnitude of the velocity difference and the head velocity signal. The conflict index was calculated from:

With the fast head motion the system lag induces a relatively large velocity difference, resulting in a conflict index of 0.81. With the slower, continuous head motions the velocity difference is smaller compared with the head velocity, resulting in a conflict index of 0.19.

The above example illustrates the importance of the interaction between the system lags in an immersive virtual environment and the characteristics of the head movements made by the users. The head motion characteristics need to be taken into account when optimising the system for a particular application.

Registration errors

The model can be used to evaluate dynamic errors in image registration, between the real world seen through a head-mounted display, and overlaid virtual objects. The figure below shows how the generic model may be adapted to evaluate image registration errors in augmented velocity tasks.

Evaluation of tracking performance and registration errors.

3.3 Recommendations for the design of applications

It might be possible to reduce side effects of exposures to virtual reality environments by either optimising the design of the system (i.e. the software and the hardware) or by implementing procedures to manage exposures. Recommendations for managing exposures to flight simulators and virtual reality environments have previously been made by Frank et al (1983) and McCauley and Sharkey (1992).

The users of virtual reality applications designed for assessment and rehabilitation may have disabilities which increase their susceptibility to certain side-effects. Many of the reported effects on performance and well-being have been ascribed to distortions in the relationship between the movements of users, and the visual feedback of those movements. These distortions arise because of limitations in the spatial and temporal performance of virtual reality systems, or because image motions are presented which give the illusion of body motion in the absence of real motion.

The proposed VRAMS modules (at least the Eating Disorders one) will deliberately create further distortions in order to augment the stimulation and feedback provided by the visual images. These distortions may be expected to give rise to additional problems. Special precautions therefore need to be taken to ensure the safety and effectiveness of such virtual reality applications. Section 4.3.4.1 makes specific recommendations for managing exposures to virtual reality environments for assessment and rehabilitation.

The design of virtual reality applications for assessment and rehabilitation may also require special precautions to ensure the effectiveness of the tasks presented to subjects. Section 4.3.4.2 makes design recommendations based on current knowledge of the effects of system characteristics on the performance of tasks and the incidence of side-effects.

3.3.1 Managing exposures to virtual reality environments

• Subjects should be informed of the possible adverse effects of exposure to virtual reality environments before they are exposed. They should be encouraged to report symptoms as soon as they occur. They should also be warned about the possible delayed onset of symptoms.
• The state of health of subjects should be screened before they are exposed to immersive or partially immersive virtual reality environments. Subjects should not be exposed if they are suffering from illnesses such as flu and ear infection, sleep loss or the effects of alcohol. Any medication subjects are taking should be reviewed for possible effects on visual or vestibular function. The postural stability of disabled subjects should also be evaluated to determine whether special precautions may need to be taken to prevent falling during or after exposures. Subjects may also be screened by a motion sickness history questionnaire (Kennedy et al, 1992) to make the experimenter aware of individuals who may have a high risk of developing simulator sickness symptoms.
• Exposure duration to immersive or partially immersive virtual reality environments should be limited to less than 5 minutes at the initial exposures. The duration of subsequent exposures may be gradually increased as the subject adapts to the virtual reality environment. Subjects should be carefully monitored during the adaptation process for simulator sickness symptoms, and exposures should be terminated when any symptoms become evident.
• The incidence of simulator sickness symptoms in a particular system, and the exposure time up to the appearance of symptoms, should be routinely logged. The incidence rate for different categories of symptoms should be regularly reviewed (Kennedy et al, 1987). This will provide an assessment of potential problem areas and allow early application of appropriate counter-measures.
• Subjects should be advised to allow a time for recovery after exposure to partially immersive or immersive virtual reality environments before engaging in potentially dangerous activities such as driving a car. Individuals experiencing postural disturbances should be monitored until symptoms have dissipated. In this case, potentially hazardous activities such as driving should be restricted for 12 to 24 hours.
  
  

3.3.2 Recommendations for the design of the VREPAR modules

The VRAMS modules that will be developed within the VREPAR project all involve some degree of manual control, or manipulation of virtual objects. The eating disorders module is a fully immersive virtual reality application in which users move through a virtual environment, observing and interacting with virtual objects. The stroke disorders module may utilise a fixed wide field-of-view display to present moving scenes which could induce a strong sense of vection. Some special considerations which should be given to the implementation of these three features are detailed below.

Interactive, immersive virtual environments

• The system should minimise the total system lag between head position sensing and presentation of an appropriate image. This may involve the selection of suitable position sensors (Meyer et al, 1992) and the minimisation of synchronisation errors (Wioka, 1995). If the total lag cannot be reduced below 100 ms thenconsideration should be given to implementing a predictive algorithm for head position in order to reduce the effects of the lag (Wioka, 1995).
• The application should be designed to require the minimum of head movements. This is particularly important during initial exposures, before the user has adapted to the altered environment.
• The maximum speed of self-movement through an immersive virtual environment should be limited to a very slow rate if it is not initiated by naturalmeans such as walking.
• Unusual and extraordinary manoeuvres, such as abruptly freezing the simulation or resetting the scene forward or backward in time, should be avoided. The scene should be removed altogether for entering and exiting the virtual environment.

Manual control or manipulation of virtual objects

• The system should be designed to have update rates of no less than 10 frames per second. With lower frame rates, subjects may adopt unnatural movement strategies, resulting in possible negative transfer of training.
• In applications where the performance of positioning tasks is critical (i.e. there is a requirement for accurate tracking or the capture of virtual objects), the total lag between limb movement and the appropriate image movement on the display should be less than 100 ms. While this is the normal design requirement it is recognised that the simulation may require lags and other distortions to be imposed on image movements. The possible effects of those manipulations require consideration (see Sections 3 and 4).

Presentation of large moving scenes in the absence of body motion

• Moving images sensed by peripheral vision can induce a strong sense of vection, and symptoms of visually-induced motion sickness. The strength of the stimulus depends the velocity of the image motion, and may be reduced if the subject fixates on fixed objects, or has a degree of control over the image motion (see Section 4.2.3.2). Particular attention should be given to the requirement to restrict the duration of initial exposures and encourage adaptation to the virtual environment.
• Disabled or unsteady individuals should be provided with adequate support to ensure they are not at risk from falling, since vection may induce inappropriate postural re-adjustments.
  

For any questions or requests, please contact auxo.psylab@auxologico.it