Virtual Reality for Health Care: a survey*

Judi Moline, Ph.D.

United States Department of Commerce

National Institute of Standards and Technology

Gaithersburg, Maryland 20899

This report surveys the state of the art in applications of virtual environments and related technologies for health care. Applications of these technologies are being developed for health care in the following areas: surgical procedures (remote surgery or telepresence, augmented or enhanced surgery, and planning and simulation of procedures before surgery); medical therapy; preventive medicine and patient education; medical education and training; visualization of massive medical databases; skill enhancement and rehabilitation; and architectural design for health-care facilities.

To date, such applications have improved the quality of health care, and in the future they will result in substantial cost savings. Tools that respond to the needs of present virtual environment systems are being refined or developed. However, additional large-scale research is necessary in the following areas: user studies, use of robots for telepresence procedures, enhanced system reality, and improved system functionality.

1. Introduction

2. Health-Care Applications of Virtual Environments and Related Technologies

3. Discussion of Current Applications

4. Areas for Further Research

5. Appendix: Medical Diagnosis and Monitoring (Telemedicine)

6. References

* (C) Judi Moline, 1997, 1998. All rights reserved.
This paper was originally published in the book "Virtual Reality for Neuro-Psycho-Physiology: cognitive, clinical and methodological issues in assessment and rehabilitation" edited by G. Riva and published by IOS Press, Amsterdam, 1997.

1. Introduction

1.1 Purpose

This paper highlights recent and ongoing research related to applications of virtual environments and related technologies in the health-care arena. The purpose is to provide an overview for those reviewing research proposals in this field. It also provides a general introduction to virtual reality especially as it relates to health care.

1.2 Background

Virtual environments and related technologies are allowing medical practitioners to help their patients in a number of innovative ways. The following are some examples:

• As the technology develops, a surgeon will be able to operate on a patient in a remote location. Today's remote telesurgery is being developed for the military to enable a surgeon to assist medics in the battle arena. The Advanced Research Projects Agency (ARPA) Advanced Biomedical Program has demonstrated its robotics surgery system using wireless transmission, with the surgical site located a kilometer away from the "robotics arms."
• Virtual environments are useful for local as well as remote surgery. An example of the local use of virtual environments is in endoscopic surgery. Surgeons manipulate instruments by viewing a television monitor and manipulating a tool inserted through a tube into the patient.
• Virtual environments are being used to create surgical simulators or trainers. These systems reduce the cost of training surgeons and the risk to patients. For example, a heart-catheterization simulation allows the trainee to guide a balloon catheter through a hollow guide wire to the obstruction, and inflate the balloon to expand the artery and restore regular blood flow.
• Therapeutic uses of virtual environments include creating interactive systems that help reduce anxiety or stress. For example, dentists are using 3-D eyeglasses to divert patients' attention while in the chair.
• Virtual environments are also used to reduce phobias, to develop skills, and to train those with disabilities. One example of the use of virtual environments for training is a program that substitutes virtual bus rides for the real thing so that disabled individuals can learn to use a public transportation system.

1.3 Definitions

Virtual environments (VEs) present a unified workspace allowing more or less complete functionality without requiring that all the functions be located in the same physical space. "Virtual environments [can be defined] as interactive, virtual image displays enhanced by special processing and by nonvisual display modalities, such as auditory and haptic, to convince users that they are immersed in a synthetic space" [1]. Less technically, "a virtual world is an application that lets users navigate and interact with a three-dimensional, computer-generated (and computer-maintained) environment in real time. This type of system has three major elements: interaction, 3-D graphics, and immersion" [2].

Satava [3] has identified five elements that affect the realism of a virtual environment for medical applications:

• Fidelity -- high-resolution graphics
• Display of organ properties -- such as deformation from morphing or kinematics of joints
• Display of organ reactions -- such as bleeding from an artery or bile from the gall bladder
• Interactivity -- between objects such as surgical instruments and organs
• Sensory feedback -- tactile and force feedback

However, realism of the virtual objects is not enough. In addition, the human-computer interaction must provide a realistic environment with which the user can interact [4].

Virtual reality (VR) is an emerging technology that alters the way individuals interact with computers. "Virtual reality is a fully three-dimensional computer-generated 'world' in which a person can move about and interact as if he actually were in an imaginary place. This is accomplished by totally immersing the person's senses...using a head-mounted display (HMD)" or some other immersive display device, and an interaction device such as a DataGlove^TM or a joystick [5]. "User immersion in a synthetic environment distinctively characterizes virtual reality (VR) as different from interactive computer graphics or multimedia. In fact, the sense of presence in a virtual world elicited by immersive VR technology indicates that VR applications may differ fundamentally from those commonly associated with graphics and multimedia systems" [6].

The head-mounted display (HMD) is a helmet fitted with paired wide-angle television screens placed in front of the eyes and stereophonic speakers placed over the ears, so that a person wearing it can see and hear only what is generated by the computer. For some medical applications, the isolating helmet has been supplanted by 3-D glasses or helmets that allow the wearer to peer at real objects by looking down rather than only at the screens in the helmet. "Several alternatives to HMDs are in the wings. The most prominent is the Cave Automatic Virtual Environment developed at the Electronic Visualization Laboratory of the University of Illinois, Chicago. The CAVE system uses stereoscopic video projectors to display images on three surrounding walls and on the floor, and the participants wear glasses with LCD shutters to view the 3D images" [7].

The DataGlove^TM is an input device (a glove that works like a joystick) that appears as a hand in the virtual world. The human user moves through the world by pointing in a direction for travel, or the user can pick up and manipulate objects by making grasping motion with the glove. A world can be anything from an abdomen to an operating room.

Telepresence surgery is performed by manipulating equipment at remote sites; image- directed surgery, on the other hand, maintains a direct connection with the real world, rather than total immersion in an artificial data space. Both involve manipulating an environment in real time, but image-directed surgery is by no means remote. Computer-assisted image-directed systems could be considered rudimentary artificial reality applications.

Partial immersion is a hybrid of digital and real environment spaces. These types of environments have been described as composite reality or augmented reality [8]. Augmented reality returns simulated cues to the operator; it is also known as enhanced reality.

Telemedicine is the real-time or near-real-time two-way transfer of medical information between places of greater and lesser medical capability and expertise. It uses technologies such as telecommunications, high-resolution graphics, imaging, and video to allow medical personnel to diagnose and treat patients without seeing them in person. Telemedicine does not use virtual environments. Telepresence systems which use a full VE for the user interface and telemedicine are often confused; Appendix A contains a brief survey of telemedicine applications.

1.4 Organization of this Paper

This paper is organized as follows:

Section 1: Introduction -- presents the purpose of the paper, background on the uses of virtual environments and related technologies, definitions of key terms, and the organization of the paper.

Section 2: Health-Care Applications of Virtual Environments and Related Technologies -- describes applications of virtual environments and related technologies in seven health-care related areas. The areas discussed are surgical procedures, medical therapy, preventive medicine and patient education, medical education and training, medical database visualization, skill enhancement and rehabilitation, and design of health-care facilities.

Section 3: Discussion of Current Applications -- addresses the value added of current applications of virtual environments and related technologies, summarizes some problems and limitations, and describes tools now under development.

Section 4: Areas for Further Research -- highlights areas were additional research is needed.

Appendix: Information on Telemedicine

References: provides a comprehensive listing of sources on virtual environments and related technologies as applied to health care.

2. Health-Care Applications of Virtual Environments and Related Technologies

The terminology and categories used in this section are an amplification of Satava's schema for health-care applications of virtual environments and related technologies [9]. The following categories of applications are discussed:

• Surgical procedures (remote surgery or telepresence, augmented -- or enhanced -- reality surgery, and planning and simulation of procedures before surgery)
• Medical therapy
• Preventive medicine and patient education
• Medical education and training
• Visualization of massive medical databases
• Skill enhancement and rehabilitation
• Architectural design for health-care facilities

2.1 Surgical Procedures

Remote Surgery or Telepresence

Telepresence applications link research in robotics and virtual environments. Telepresence systems are used in medicine to manipulate equipment at remote sites. The surgeon or other medical practitioner has the sense of actually being at the site performing the procedure [10].

SRI International's Green Telepresence Surgery System was designed to allow surgeons to participate in battlefield operations from sites removed from the front line. The system consists of the "remote operative site and a surgical workstation...[that includes] 3-D vision, dexterous precision surgical instrument manipulation, and input of force feedback sensory information." The surgeon operates on a virtual image, and a robot on the battlefield reproduces the surgeon's movements. This one-to-one coupling between the surgeon and the machine has been demonstrated to work from 150 yards away with a fiber-optic connection. The next goal for the military is to replace the fiber-optic connection with a wireless signal [11]. As promising as this system is, it will take about 3 years for it to be approved for investigational trials on humans [12].

Recently, supervisory staff at medical schools such as Johns Hopkins have been using telepresence. The supervising staff member remains in his/her office and holds a laparoscope remotely for the surgeon. The remote interaction is facilitated by two monitors and a hard-wired connection [13].

The use of virtual environments and remote surgery opens up new possibilities. Besides the military and the academic examples discussed above, other civilian uses are possible. A specialist could assist a local surgeon by remote connection, or surgery could be performed, via a medical center, in a rural setting, a ship at sea, an airplane in flight, or even a space station. Besides solving the distance problem, telepresence offers other benefits, such as minimizing the exposure of surgeons to diseases and reducing potential costs as a result of reduced trauma [14].

The military surgery system and academic application described above fit the narrow definition of telepresence or remote surgery as the manipulation of equipment at a remote site. However, many of the procedures described in the following subsections will lend themselves to robotics telepresence in the future.

Augmented -- or Enhanced -- Reality Surgery

Augmented -- or enhanced -- reality surgery is being used for minimally invasive surgery. Augmented surgery includes fusing computer images with real time images or superimposing them on the body using video technology or robots directed by surgeons physically present in the operating room. Table 1 shows some applications of these techniques.

Table 1. Applications for Augmented Reality Surgery

Traditionally, surgery is performed by making incisions and directly interacting with the organs and tissues. Recent innovations in video technology allow direct viewing of internal body cavities through natural orifices or small incisions. As with remote surgery, the surgeon operates on a virtual image. The manipulation of instruments by the surgeon or assistants can be direct or via virtual environments. In the latter case, a robot reproduces the movements of humans using virtual instruments. The precision of the operation may be augmented by data/images superimposed on the virtual patient. In this manner the surgeon's abilities are enhanced.

Surgical practice, particularly in orthopaedics, presents excellent opportunities for robotic and computer-based technologies to improve clinical techniques. Procedures such as total joint replacements are performed in large volumes and at significant cost. The clinical success of these procedures is very dependent on the proper placement and fit of the implants within bony structures. Important contributions to surgical planning and execution can be made by surgical robots and pre-operative planners that utilize computer simulations. Use of robotic assistants significantly augments the skill of the surgeon. [15].

In 1989, video technology was employed by a team of surgeons to perform a laparoscopic cholecystectomy (removal of a gallbladder). The use of this minimally invasive therapeutic technique has become standard: "about 90% of all cholecystectomies performed in the last two years have been done using laparoscopic techniques" [16]. Use of endoscopes (instruments that use video techniques for visualizing) are now commonly used for surgery.

In 1993 surgeons at Brigham Women's Hospital in Boston, "with help from engineers from General Electric Co.'s imaging and visualization laboratory in Schenectady, N.Y., ... began modeling work to assist operations in real time. As of early November [1993], 17 operations had been performed" with the surgeons using a monitor. In one example, a magnetic resonance image (MRI) taken earlier was overlaid on a real time video image of the patient's head. [17]. This technique can provide an X-ray view of a tumor that might otherwise not be visible if it is deeply embedded in the brain tissue [18]. Similarly, "UNC, Chapel Hill, is overlaying ultrasound images on live video that is then viewed in an HMD [head-mounted display]. [Again], this system essentially gives the viewer the feeling of having X-ray vision." Aligning images in real time is an area for major research [19].

A low-cost interactive image-directed neurosurgical system is being used in epilepsy surgery. The system has been successfully used in 20 epilepsy surgery cases. The epilepsy surgery is directed by the physiological and functional data obtained during the preoperative evaluation. Computer-assisted image-directed techniques are particularly well-suited for this surgery. "An image-directed system melds the preoperative data to the surgical anatomy by defining all the relevant data topographically in 3D-space, and coregistering this data to the anatomical image and the patient space" [20].

"Dr. Henry Fuchs of the University of North Carolina has created a [virtual reality] model which allows the physician to visualize the tumor (from reconstructed 3-D CT scans) inside the individual patient and plan various radiation trajectories to allow lethal doses of radiation to the tumor while avoiding damage to normal organs" [21].

Additionally, magnetic resonance "allows the visualization of temperature changes for 'thermal surgery' procedures, a capability never previously available non-invasively. For the first time, 'real-time' 3D images of temperature changes can be obtained. As well, [magnetic resonance] images show a 'phase change' upon heating, and it is postulated that this occurs when the tissue protein is denatured (i.e., when coagulation takes place). These two capabilities provide MR with the ability to monitor and control thermal therapies. Tissue ablation can be monitored during the deposition of energy via interstitial laser or focused ultrasound therapy, RF ablation or cryosurgery" [22].

Planning and Simulation of Procedures Before Surgery

Virtual environment technology can also be used to improve the way surgeons plan procedures before surgery. When used to simulate a procedure on a particular patient, the technology can be used to integrate the information provided by diagnostic sensors into a realistic model of the actual environment in which the surgeon will perform the proposed procedure [23]. The realistic model or surgical simulator "must have accurate detail and must be highly interactive. Specifically, the image must be anatomically precise and the organs must have natural properties such as the ability to change shape with pressure and to behave appropriately in gravity. All the body parts represented must be able to be manipulated by grasping, clamping, or cutting, and they must be able to bleed or leak fluids" [24].

Virtual environments can make a critical contribution to the planning of a surgical procedure (see Table 2). For example, "Netra has been used for various precision, computer-assisted surgical procedures. Neurosurgeons use Netra to plan precision biopsies, laser-guided tumor resections, surgery for Parkinson's Disease and other motor disorders, and surgical implantation of electrode arrays for epilepsy" [25]. What is unique with this system is its user interface. Users manipulate objects, such as a doll's head, to cause movement of the virtual image on the monitor.

Simulators are being developed for all types of surgery. Many of them are used for planning particular procedures. Dr. Altobelli's (Brigham Women's Hospital) system "creates 3-D images from the CT scan of a child with bony deformities of the face (cranio-facial dysostosis); using the model created from the CT scan, the bones can be correctly rearranged to symmetrically match the normal side of the face." The procedure can be practiced repeatedly [26].

Table 2. Applications for Planning and Simulation of Procedures Before Surgery

Some of the simulations designed for planning a procedure also include a predictive component. Data that could be used to correct a condition is introduced into the model and the results of the proposed actions are calculated and visualized. For example, Dr. Joseph Rosen's (Dartmouth University Medical Center) "[virtual reality] model of a face with deformable skin...allows the practicing of a plastic surgical procedure and demonstrates the final outcome." Another example is Dr. Scott Delp's "virtual model of a lower leg upon which he can practice a tendon transplant operation and then 'walk' the leg to predict the short and long term consequences of the surgery" [27].

An application allowing evaluation of the probable results of a procedure is work from the Fraunhofer-Institute for Industrial Engineering and the Orthopaedic University Clinic Heidelberg. They have "developed an application for the planning of osteotomy operations. The surgeon, with the aid of tracked shutter-glasses and a 6D input device, is able to view and rotate the femur and the hip joint." Using a simulated X-ray of the bones, the medical practitioner can determine the angle, allow the computer to perform the surgery, and view the simulated results on the screen [28]. Planning for surgery is always important. With computer-assisted surgical planning, an exact picture can be drafted "of the relative positions of the cartilages, the abrasion of the cartilage and the position of the femur in the hip joint, [to help] determine the alteration parameters. The planning of an osteotomy can be optimized by using a 3D image because the operator is more able to evaluate the result. An increase in the rate of successful operations is conceivable" [29]. In the future, the application should serve in the planning of patient-specific osteotomies requiring tomographic data from the relevant bones and cartilage [30].

Simulation techniques for improved preoperative planning are of value for orthopedic surgery with or without a robot. It is

...useful for a surgeon to realistically simulate surgery beforehand to help determine what implant size to use, and what would be its optimal position. Existing planners permit the surgeon to template the implant based upon 3D geometry, but give no indication of the consequences of the proposed surgery on the initial stability of the system, the presence of implant-bone interface gaps, and the changes in the mechanical environment that are induced in the bone. The inclusion of biomechanical simulations would permit the surgeon to make appropriate changes in the initial surgical a plan, changing such parameters as the implant placement, specifics of the bone preparation, and type and size of implant. Without a robotic tool, however, there is no method for a surgeon to accurately implement a preoperative plan. For example, the simulation may help indicate an 'optimal' bone cavity shape and implant location, but the surgeon will be unable to accurately perform this plan without a robotic device. In this manner, surgical robots actually improve the clinical usefulness of realistic surgical simulations [31].

The predictive element is important not only for the medical team, but also for the patient's family. It can serve as a model on which to base informed consent of the patient and his or her family [32]. Further, these simulation applications could be used for medical education and training.

2.2 Medical Therapy

Virtual environments are being used for a variety of purposes categorized here as medical therapy (see Table 3). These include treating eating disorders, phobias, and other psychological conditions, diverting patients' attention during medical intervention sessions, providing an appropriate environment for physical exercise and/or relaxation, and providing a means for patients to describe their experiences from within altered states.

Recent research by Giuseppe Riva has shown the efficacy of the Virtual Environment for Body Image Modification (VEBIM) theoretical approach. This approach focuses on the treatment of a disturbed body image in order to resolve eating disorders. Preliminary results show that after the VR experience, the subjects were more satisfied with their actual body image (i.e., less intent on achieving an ideal body image) [33].

After suffering a stroke, patients provided with stimulation interventions attain some degree of recovery of function. The system being developed translates bio-feedback to action-feedback through a virtual environment. By providing patients with visual information about specific muscular activity, it is expected that they will make significant but limited gains [34]. Similarly, a virtual reality experimental system is being developed to study motion in some diseases of the basal ganglia, e.g., Parkinson's disease. The aim of this research is to develop and use immersive virtual environments to study the relationship between visual sensory information and control of movement being performed by those suffering from these diseases [35].

Since the spring of 1993, researchers at Georgia Institute of Technology's Graphics, Visualization, and Usability Center have been exploring the possibility of using virtual environments for therapy of individuals with psychological disorders. One of their research areas has concerned acrophobia (fear of heights). The researchers' goal is to

lessen a person's anxieties through practice in well-controlled situations. One of the main advantages of a virtual environment is that it can be used in a medical facility, thus avoiding the need to venture into public situations [36]. Ralph Lamson of Kaiser-Permanente, San Rafael, California, is using a similar system for virtual therapy [37]. Another application of virtual reality to psychology helped in training autistic children. The study was designed to replicate skill training done by therapists. The goal was to train a subject to find an object in the environment, walk to it, and stop. The two autistic children used as subjects accepted the technology and responded to the learning situation [38].

Table 3. Applications for Medical Therapy

Another trend is the use of virtual environments in medical edutainment. Dentists are using virtual environments to divert patients' attention while in the chair. Using 3-D glasses with headphones hooked up to a tuner and VCR, patients can watch 2-D and 3-D videos or educational programs, listen to music, and play video games [39]. Another example is its use in a medical support program for cancer patients. Cancer patients may experience insomnia and unrest, especially when undergoing chemotherapy. The purpose of the Psycho-oncological VR Therapy (POVRT) application is to diminish these psycho-oncological problems through virtual environments. In the virtual space, the patients can feel as if they are outside the hospital. Further, "some patients memorize the experience of having had nausea and vomiting when they were first treated with chemotherapy, and they complain in the second chemotherapy [session] even before the treatment" starts. Such anticipatory emesis may be reduced with virtual environment applications [40].

A stationary bike that allows the user to race against the computer or other people, changing gears, braking, and tilting the seat to steer has been developed. A variety of on-screen terrains offers different degrees of pedaling difficulty; going faster causes the feeling of simulated wind resistance [41]. Another product is a "massage chair that uses [virtual reality] images and sound to help relax users; it monitors pulse rate and automatically controls images and sound to match the user's body condition and state of consciousness" [42].

Still to be developed is the concept of using virtual reality as a medical and patient education and evaluation tool. The idea is to allow a virtual reality experience to be built from within altered states, including hemianopia, proprioceptive disorientation, or muscular damage [43].

2.3 Preventive Medicine and Patient Education

The areas of preventive medicine and patient education lend themselves to virtual environment applications. Video games are not necessarily virtual environments although some use VEs. VEs could become an important educational tool, as well as provide edutainment. For example, a video game series (Health Hero^TM) is being developed for pediatric patient education. The video games are expected to enhance young people's self-esteem; self-efficacy; communication about health; health knowledge and skills; and motivation to learn about health, and how these factors can influence self-care behavior and health outcomes [44].

This is an area that will benefit from work done in other areas of virtual environments, although to be effective, products must be tailored to the particular applications and users.

2.4 Medical Education and Training

Medical Education

Virtual reality allows information visualization (the display of massive volumes of information and databases). Through 3-D visualization, students can understand important physiological principles or basic anatomy [45]. Table 4 lists some applications of virtual environments and related technologies for medical education. In this paper, education is used for applications that do not allow the learner to practice a procedure while training is used where practice is the objective of the application.

One application for the study of anatomy allows the abdomen to " be explored by 'flying' around the organs (a bird's-eye view from the diaphragm) or behind them (watching the bile duct pass through the pancreas under the duodenum) or even inside them. (A trip might start from inside the esophagus, down the stomach, through the duodenum, up the bile duct, and out the gall bladder is a popular route [sic].)" The extraordinary perspectives provided by such a learning tool "impart a deeper understanding and appreciation of the interrelationship of anatomical structure that cannot be achieved by any other means, including cadaveric dissection" [46].

Virtual environments provide "both a didactic and experiential educational tool. A demonstration mode could give a 'tour' of the intended subject, and then an exploration mode would allow the student to actually experience the environment" [47].

Table 4. Applications for Medical Education

Dr. Helene Hoffman of the University of California, San Diego, is working to create 4-D, " the three dimensions of a virtual world (3-D space) and the fourth dimension of time (archived information in multimedia format), in essence, multimedia virtual reality (MMVR). For example in a MMVR simulator of the gastrointestinal tract, a student could 'fly' down into the stomach, see an ulcer and 'grab' it as if for a biopsy, this would bring up the histologic micrograph of an ulcer, or play a video tape of a Bilroth 2 operation for ulcer disease, or perhaps demonstrate (predict) the healing in response to medication. In this fashion the multiple layers of understanding could be rolled into one, and the change of the processes over time can be graphically represented and personally experienced" [48].

The following are additional examples of systems being used in education or training:

• A German system for orthopedic surgery is being used for educational purposes with data records procured from the company Viewpoint DataLabs. "The data records are models from people with average body structure and are therefore suitable for general educational purposes" [49].
• A computing system for visualizing and monitoring human labor and birth, from the Health Sciences Center at Brooklyn, New York, offers an "opportunity for the study of large numbers of collected cases, and for modeling scenarios for the purpose of education and research. Particularly instructive labors can be preserved in a computerized 'teaching file,' and become objects of instruction for future students in labor" [50].
• The Center for Human Modeling and Simulation at the University of Pennsylvania is developing models of functional anatomy. The center is first modeling the respiratory mechanism since it involves physiological change, such as pressures and flows, that depends on gross anatomical deformations. Ultimately, using models for other physiological systems, the center will demonstrate the interactions between systems due to the physical space they share.
• The opportunity for international participation in medical conferences is another potential application currently being explored [51]. Interaction and participation using virtual environments might alleviate some of the stiltedness of current videoconferencing capabilities. This use of a shared virtual environment instead of real time transmission suggests an application of VEs to telemedicine. (See the Appendix for applications of telemedicine.)

Medical Training

The use of computers in medical schools is increasing. Table 5 lists some examples of applications of virtual environments and related technologies for medical training.

In medical research, there is a shortage of cadavers; for students, being able to summon a detailed, lifelike computer image of a human body is a workable replacement. To improve training, companies are developing 3-D computer simulations of medical and surgical procedures [52]. According to Jon Merrill," these 3-D simulations allow rather more realistic training than do cadavers... . If a trainee pinches a nerve or slices through a blood vessel in a cadaver, nothing happens. Dead tissue is usually harder, color is changed, and arteries no longer pulsate. Once something is cut in a cadaver, it cannot be reattached. But in digital format, procedures can be repeated ad infinitum and stored for later group analysis. Some organs can also be made transparent, 'crisis' drills can be created, and various types of organs can be modeled" [53].

Table 5. Applications for Medical Training

A system built at Talisman Dynamics, Inc. simulates the rather rare open cholecystectomy procedure. Because the laparoscopic procedure is being addressed by a variety of applications for medical training, Talisman felt that a simulation of the open procedure would lend itself to an overview of the abdominal anatomy in 3-D context that could be useful in anatomical education. Simulating the open procedure allowed the developers "to simulate a large number of different tools and interactions, and to provide a comprehensive spatial context" [54].

High Techsplantations' initial work involved the development of a "virtual abdomen." The virtual abdomen or "uro man" could be rotated using a three-dimensional mouse. After a year of research and development efforts, a laparoscopic lymph node dissection simulation was completed. In parallel, High Transplantations "simulated other procedures, including an angioplasty procedure (sponsored by Marion Merrell Dow). This simulation allows the user to use a simulated balloon catheter to practice angioplasty. The software allows for various complications including transection of the coronary vessels, rupture of the balloon, as well as

resistive feedback to the end-user. A specially designed catheter, equipped with position sensors and feedback devices was constructed to afford a high-fidelity tactile simulation" [55].

"Woods and Hon have developed virtual reality laparoscopic surgery simulators. These consist of a simple plastic torso into which the handles of laparoscopic instruments are mounted (to provide force feedback); the virtual abdomen (liver and gall bladder) are graphically demonstrated upon the video monitor, and the apprentice surgeon can practice the specific laparoscopic procedure" [56]. Satava took a different approach. "A virtual abdomen has been created for the immersive, traditional helmet mounted display (HMD) and Dataglove^TM. Using virtual scalpel and clamps, the abdominal organs can be operated upon. This same abdomen can be 'explored' by a student in the manner described above [57].

The following are examples of additional issues being addressed by simulators recently or currently under development:

• A limb trauma simulator is being developed and is expected to lead to a virtual reality environment [58].
• Georgia Institute of Technology in Atlanta is trying to simulate surgery on the human eye, complete with feedback on the force being exerted [59].
• The National Institute for Cancer Research in Genoa, Italy, has developed a virtual reality microsurgical simulator for surgeon training. They want to create a microsurgical training simulator that simulates the suturing of two parts of a vessel under a stereo microscope.
• The National Cancer Center of Tokyo has developed a Surgical Simulation Support System. The surgical procedure simulated is a neurosurgical operation in which a neoplasm of the brain is resected without photomicroscopy [60].
• Researchers at the Ohio State University Hospital, Immersion Corporation, and the Ohio Supercomputer Center are collaborating "to create and test a virtual simulator for training residents in the use of regional anesthesiology. Specific issues and difficulties of the epidural technique were used to develop a pilot system... . Limitations of physical models such as mannequins include lack of patient variance, inaccurate representation of biological tissue, and physical wear from repeated use" [61].
• "Triage is a protocol to assess patient conditions and to decide on medical treatment in mass casualty situations. Education and training on triage protocols can be facilitated with computer-based training facilities, like interactive video. [Virtual environment] training and simulation systems can give the human an experience which is near to reality... , In a [virtual environment] training system, mass casualty triage in combat situations can be simulated. There is, however, some more research required towards deformation modeling and real-time database management. Present display devices have a resolution in terms of image size that is critical to medical applications" [62].

In addition to building simulators for specific procedures, generic problems are being addressed, such as the introduction of olfactory stimuli in virtual environments. Surgical simulations need to "provide the proper olfactory stimuli at the appropriate moments during the procedure. Similarly, the training of emergency medical personnel operating in the field should bring them into contact with the odors that would make the simulated environment seem more real and which might provide diagnostic information about the injuries that simulated casualty is supposed to have incurred" [63].

Another generic issue is the development of low-cost systems for mass distribution. "[Virtual reality] surgical simulators will someday be valuable in medical education, both to reduce training costs and to provide enhanced physician training." However, "many surgical simulators rely on real-time volume rendering techniques, and run on hardware costing several hundred thousand dollars." The Departments of Surgery and Computer Science at Stanford University "have created a low cost, interactive [virtual reality] application for the IBM PC that provides a much clearer perception of surgical anatomy than a surgical atlas while remaining comparable in price. This program could be used to teach hundreds of other surgical procedures" [64].

2.5 Visualization of Massive Medical Databases

"Virtual reality is a way to visualize, manipulate, and interact with computers and extremely complex data. Interactive image-directed neurosurgery may be considered a primitive form of virtual reality since it essentially does this in the context of performing surgery. With interactive computer image-directed techniques, the neurosurgeon navigates through anatomic, functional, and physiologic 3D space defined by digital images obtained from and representing the patient. The image space is a virtual reality data set representing functional as well as anatomic digital data" [65]. The intent of this category is to use virtual environments to symbolically represent data as visual objects in a virtual environment. Table 6 lists some applications of virtual environments and related technologies for visualization of massive databases.

Table 6. Applications for Visualization of Massive Medical Databases

Under funding from the Department of Defense, the Ohio State University Hospital's Department of Anesthesiology is "creating a system for teaching a specific method of regional anesthesia, the epidural technique... . The system will enable the resident to investigate various three-dimensional reconstructed data sets in a non-threatening environment. The system can be cued through voice activation to provide additional information in text, audio, or graphical form. Furthermore, the system incorporates the necessary components to allow the resident to 'feel' the technique as performed by the expert" [66].

"Henderson has created a cyberspace representation of the war injuries from the Viet Nam Database. Using a 3-D cube to plot 3 axes of information, complex combinations of war wounds, organ systems injured, mortality, etc. can be visualized as clusters of data points. These clusters can illustrate and reveal important relationships which otherwise cannot be discovered. Navigating in three dimensions allows different perspectives of the data, permitting different interpretations. This application for visualization has not been exploited sufficiently and holds promise for the field of medical informatics" [67].

2.6 Skill Enhancement and Rehabilitation

Applications of virtual environments and related technologies for skill enhancement and rehabilitation include those that provide training in the use of equipment, those that allow the exploration of virtual space, those that augment physical abilities, and those that teach skills. Table 7 lists some examples.

"Dean Imman at the University of Oregon is using a [virtual environment] to train disabled children to control motorized wheelchairs. For a child who is learning to use a motorized wheelchair but who has limited control over his body, for instance, a [virtual environment] overcomes several potential safety problems. This [virtual environment] training might also be motivational, because children accustomed to constant care by others can feel threatened by a wheelchair's mobility" [68].

The use of virtual environments for rehabilitation in medicine has grown dramatically, resulting in an annual conference devoted to the subject. "Greenleaf Medical Systems has created virtual environments for exploration in a wheelchair. For example, an individual with cerebral palsy who is confined to a wheelchair can operate a telephone switchboard, play handball, and dance within a virtual environment" [69]. "Warner has utilized an eyetracker device from BioControl, Inc. in a quadriplegic child in an effort to provide her the opportunity to develop interactions with the outside world before her disability causes her to become too introverted to communicate. In these circumstances, [virtual reality] is being used to empower those individuals with disabilities" [70].

The visually impaired have difficulty using conventional computer display products. Current products for the visually impaired limit the amount of enlarged imagery to as little as 1 percent of a document page. Further, it is difficult to determine the cursor position on the page. "A prototype system was designed using a virtual computer monitor (VCM) that moves the user's line of sight across an enlarged virtual document, instead of vice-versa, using a virtual reality head-mounted display (HMD). The wearer's head position is sensed using a head-tracking device, and translated into mouse output in software. The simulated mouse data is used to scan the enlarged computer output across the HMD field-of-view in the opposite direction from the wearer's physical head movement, causing the impression of moving one's view about a fixed, enlarged document" [71].

The University Applied Physics Laboratory at The John Hopkins University, Baltimore, Maryland, is working on an Interdivisional Sensory Engineering Program in cooperation with the Schools of Medicine, Engineering, and Arts and Sciences for the use of virtual reality to enhance vision of the visually impaired [72].

"A primary symptom of Parkinson's Disease...progressively restricts the ability to walk... . Virtual reality technology provides a way...to enable walking, by presenting virtual objects overlaid on the natural world. Normal stride length, and even running, can be enabled, provided the virtual clues are spaced properly and simulate the appearance of staying stable on the ground as the person moves" [73]. Glasses displaying a television image "help Parkinson's disease patients overcome the halting, hesitant gait that characterizes the disease... . The...glasses project a track of objects, currently yellow blocks, at stride-spaced intervals. Wearers step across the track, facilitating nearly normal walking patterns" [74].

Table 7. Applications for Skills Enhancement and Rehabilitation

"The Train to Travel project, sponsored by the University of Dayton (UD) Research Institute in cooperation with the Miami Valley Regional Transit Authority (RTA), substitutes [virtual reality] bus rides for the real thing. Students learn independently in the classroom, eliminating the need for a teacher to accompany or follow them on real trips" [75]. "In the project, students use interactive multimedia to recognize landmarks and learn what to do in case of an emergency. When they master basic skills, they progress to the virtual reality environment, where they use an [head mounted display] system with head tracking to look around a computer-generated landscape" [76].

2.7 Architectural Design for Health-Care Facilities

Virtual environment testing of architectural designs for health-care facilities could save both time and money. However, considerable research on interaction methods and software system design will be needed before those savings can be realized [77]. To illustrate what is being done, "Dr. Kenneth Kaplan of the Harvard Graduate School of Design is beginning to apply [virtual reality] to architectural design in a project for the operating room (OR) of the future" [78].

In one example of using a virtual environment for design purposes, "a person in a wheelchair puts on a Virtual Research helmet equipped with electromagnetic motion trackers. Objects are manipulated via a Dataglove... . The architect can point to representation of a window sill, for example, and make it higher or lower, or widen a door if it's too narrow" [79].

Companies are also using virtual reality to sell their health-care products. For example, one "has adopted a relatively low-cost, custom-built system to present interactive layouts of its healthcare equipment to potential clients in realtime" [80].

Table 8 lists examples of applications of virtual environments and related technologies for architectural design for health-care facilities.

Table 8. Application of VE and Related Technologies in Architectural Design for Health-Care Facilities

3. Discussion of Current Applications

This section presents a discussion of the current applications of virtual environments and related technologies described in Section 2. It summarizes the value added of these applications for health care, notes some problems and limitations, and describes some tools currently under development.

3.1. Value Added

Virtual environments and related technologies add value to health care in the areas of cost savings, improved services, and savings in material resources. Table 9 summarizes examples of value added in these three areas.

Cost Savings

Trauma units in emergency rooms could improve operating efficiency and reduce costs by using specialists who are not physically present. With telepresence, experts could be linked to remote patients. This is effective whether distances are great or not. Because the need for specialists in emergency situations cannot be predicted, making use of remote specialists in time of need limits the staffing needs of the trauma units without limiting their effectiveness. Doing this would conserve resources by limiting the need for part-time specialists to be physically present in trauma units.

Improved Services

Virtual reality is impacting and improving surgical results. Examples include the use of laparoscopic simulators for training, the development of applications that simulate human response to medication (for example, simulator systems helping to train anesthesiologists), and the development of imaging tools that guide surgical tools through brain tissue to the site of a tumor [81].

Table 9. Value Added to Health Care by Virtual Environment Systems

Advantages offered by telepresence systems include enhancing task performance in remote manipulation through increased positioning resolution or range; allowing controlled application of extremely large or small forces; improving operator perception of the task; and facilitating manipulation in hazardous environments by isolating the environment from the operator, or manipulation in clean environments by isolating the operator from the environment. Common to all telepresence systems is a human operator in a manual or supervisory control loop overseeing task performance. Application areas include operations in radioactive, underwater, space, surgical, rehabilitation and 'clean-room' environments, as well as the manufacturing and construction industries.

Non-Renewable Resource Savings

Finally, the use of simulators saves precious resources such as cadavers and animals. By allowing medical personnel to train using simulators, the demand for non-renewable resources can be drastically reduced. The trainee can practice over and over using a realistic, virtual environment without reducing the supply of non-renewable resources.

4.2 Problems and Limitations of Current Applications

Many of the current virtual environment applications in health care have problems that limit their effectiveness. Some of these limitations are due to the state of the art of the supporting technologies.

As noted earlier, the sense of smell in virtual environment systems has been largely ignored. Both Krueger and Keller are developing odor-sensing systems [82]. Smells are extremely important. Not only do they help distinguish specific substances, but also they give a sense of reality to a situation. The absence of odor is a serious limitation of current telepresence and training systems.

Another major research problem relates to overlaying ultrasound images on live video that is then viewed in a head-mounted device application. The research issue to be addressed is the alignment of images in real time [83].

"The senses of vision and touch are the two main feedback mechanisms used by surgeons when performing a surgical procedure. Improved cameras, HDTV, head mounted displays, and stereoscopes have advanced the sensing and displaying of vision; however, there have been few developments in the area of tactile feedback. The ability to feel tissue is a valuable tool. Procedures that require palpitation, such as artery localization and tumor detection, are extremely difficult when the only form of haptic exploration is in the form of forces transmitted through long, clumsy instruments. The ability to remotely sense small scale shape information and feel forces that mesh with natural hand motions would greatly improve the performance of minimally invasive surgery and bring a greater sense of realism to virtual trainers" [84].

"According to experiments conducted by the British Defence Research Agency virtual reality can make you sick... . [This confirms] what some users of [virtual reality] headsets have long suspected. Researchers found that 89 of 146 otherwise healthy adults suffered temporary nausea, dizziness, or impaired vision after using a [virtual reality] helmet-mounted display (HMD) for just 20 minutes. Eight became so nauseated they could not finish the 20-minute period... . Virtual reality researchers attribute the apparent effects to a couple possible causes. First, [virtual reality] systems usually have a lag time between when the user moves and when the display is updated; users, accustomed to this lag, may become confused when they take the helmet off. Moreover, [virtual reality] creates an illusion of three dimensions, even though the screen actually remains at a fixed distance from a wearer's eye. This may disrupt depth perception later" [85].

Today simulations trade off less realism for more real-time interactivity because of limited computing power, but the future holds promise of a virtual cadaver nearly indistinguishable from a real person. "Initial research over the past 5-10 years in video technology, graphics, computer-aided design (CAD) and virtual reality has given us an insight into some of the requirements for a virtual-reality surgical simulator. If graphic images are used, it is estimated a rate of 5,000,000 polygons/s would be required for a realistic reconstruction of the abdomen; current high-level computer-graphics workstations generate 60,000-100,000 polygons/s. Using CT or MRI scan images would require substantially more computer power. The algorithms for object deformation and gravity are available and continue to evolve. For motion, at least 30 frames/s are necessary to eliminate flicker and response delays; this level of interactivity is available on standard [virtual reality] systems. The computational power for sensory input and for object reactivity has not been determined" [86].

Fujita Research has created a situation in which a construction robot in America can be controlled from Japan. If it can be done with a construction robot, it will not be long before this is applied to telepresence surgery. "The biggest problem now is the time delay involved in long distance communications. Currently, there is a delay of a second or more from the time the command is given until the robot actually moves. The same is true when sending a camera image from the robot. Fujita hopes to cope with the delay problem by employing computer simulations of the robot movements and displaying predicted motions on the operator screen" [87].

Virtual reality systems used for interfacing to "training equipment require the user to be isolated from the real world and surrounded by the virtual world generated by the computer. This is normally achieved using a head mounted display...enclosed in a rugged casing" [88]. One of the challenges for virtual environments is to allow the user to move freely between the virtual and the real worlds.

4.3 Tools Under Development

There are many types of tools under development that will facilitate the use of virtual environments in health care. These tools provide computer input, computer output, or both. Examples of these are listed in Table 10.

"David Warner at the Human Performance Institute of Loma Linda University Medical Center, California, is employing [virtual environment] technologies to provide greater interactivity to people with severe disabilities. He has developed computer interfaces that use eye movements and muscle potentials. Severely disabled people have been able to navigate through [virtual environments] using biological signals from muscles over which they retain control" [89].

"Researchers at the Systems Research Laboratory of the Advanced Telecommunications Research (ATR) Institute in Kyoto are working on a system that uses image processing to recognize gestures and facial expressions as input. Such a system would provide a more natural interface without requiring the operator to wear any kind of special gear" [90].

Hiroo Iwata of the Institute of Engineering Mechanics at the University of Tsukuba "has a virtual perambulator under development. In this project he wants to build an apparatus that will take the previously separated bodily sensations and handle them together. The perambulator uses a head-mounted display. The walker's upper body is fixed in place, and a goniometer detects the position of the head. Ultrasonic wave generators are attached to the walker's toes. The time required for the ultrasonic waves to reach receiver units determines the positions of the walker's feet. These motion data are sent to a computer that generates a virtual space, and the view through the HMD changes in real time in response to the walker's movements... . The perambulator is made so that the tension on wires attached to the feet produces the sense of reaction or resistance associated with climbing or descending stairs. When the walker ascends a step, the wire length is regulated so that the take-off foot feels the force of resistance. To represent the reaction force involved in opening a virtual door, the virtual perambulator includes a manipulator having six degrees of freedom. 'With this apparatus,' said Iwata, 'bodily movement is coordinated with visual images. A more lifelike model can be experienced'" [91].

According to Kirby Vosburgh, manager of the General Electric's Imaging and Visualization Laboratory, "GE is working on real-time MRI and CT scans for the operating room. If that succeeds, surgeons could have high-quality views of the body's interior as they worked" [92].

The ability to feel objects in virtual environments can markedly enhance the effectiveness of many applications, particularly for training, scientific visualization, and telepresence. "Haptic displays, devices for presenting tactile and force sensations, are being developed in several laboratories, but are not yet widely used elsewhere. Most of the haptic displays being developed are electro-mechanical devices that deliver force feedback to the hand or arm within limited ranges of movement. For example, Makato Sato of the Tokyo Institute of Technology's Precision and Intelligence Laboratory has developed a force-reflecting system called Spidar (Space Interface Device for Artificial Reality). With this system, the user inserts his or her thumb tips and index finger into a pair of rings, each of which have four strings attached to rotary encoders. The encoders are located at the corners of a cube. String movements can be restricted with brakes, providing touch sensations. Spidar is being applied in a [virtual environment] for collaborative design" [93].

Table 10. Examples of Tools for Use in Virtual Environments in Health Care

The PHANToM^TM actively exerts an external force on the user's finger(s) creating the illusion of interactions with solid virtual objects. Users can actually feel the tip of the stylus touch virtual surfaces. Surgeons can practice procedures on virtual patients. "The Phantom consists of a finger thimble mounted on the end of a small robot arm. When the thimble hits a position corresponding to the surface of a virtual object in the computer, three motors generate forces on the thimble that imitate the feel of the object... . The Phantom can duplicate all sorts of textures, including coarse, slippery, spongy, or even sticky surfaces. It also reproduces friction. And if two Phantoms are put together a user can 'grab' a virtual object with thumb and forefinger." Researchers are using the PHANToM^TM for surgical training, prototyping, and drug design [94].

Other force feedback tools include a system under development at the University of North Carolina, Chapel Hill, which "uses an Argonne Remote Manipulator (ARM) to provide force feedback to the arm for molecular modeling applications. Hiroo Iwata of the University of Tsukuba has developed a pen-shaped, 6-degree-of-freedom, force-reflecting device. A 4-degree-of-freedom, force-reflecting manipulandum has been developed by Northwestern University" [95].

Recently, Hiroo Iwata of the Institute of Engineering Mechanics at the University of Tsukuba has developed " 'volume haptization' techniques that represent higher dimensional scientific data by force sensation. The system maps voxel data to the three dimensions of force and three dimensions of torque in the force display. The operator thus feels the visual image of column data on the screen" [96].

Addressing the issue of incorporating the sense of smell in virtual environments, Keller et al. have developed a "chemical vapor sensing system for the automated identification of chemical vapors (smells)... . An electronic nose will potentially be a key component in olfactory input to a telepresent virtual reality system. The identified odor would be electronically transmitted from the electronic nose at one site to an odor generation system at another site. This combination would function as a mechanism for transmitting olfactory information for telepresence. This would have direct applicability in the area of telemedicine since the sense of smell is an important sense to the physician and surgeon" [97].

"[Engineer Michael] Burrow [of Georgia Institute of Technology] and his colleagues at the Medical College of Georgia are developing a 'teletactile hand' that will someday enable doctors to electronically reach out and touch someone in remote areas with few medical specialists" [98]. A related project at Georgia Tech uses tactile feedback to help students practice surgery [99].

Practicing Mayo Clinic surgeons are committed to assisting with the development, evaluation and deployment of the VRASP system discussed earlier. "VRASP will bring to the [operating room] all of the pre-surgical planning data and rehearsal information in synchrony with the actual patient and operation in order to optimize the effectiveness of the procedure, minimize patient morbidity, and reduce health care costs" [100].

Modules of the Virtual Reality All-purpose Modular System (VRAMS) being developed as part of the Commission of the European Communities (CEC) VREPAR Project are being used to study and treat eating, stroke, and movement disorders. The goal of this demonstrator of a virtual reality system is to provide for psycho-neuro-physiological assessment and rehabilitation [101].

4. Areas for Further Research

Virtual environment applications for health care are currently available and are being used to achieve the benefits discussed in Section 3. The demands of surgery, training for surgery, and telepresence surgery require a high degree of realism as concerns transmitted images. Additional large-scale research is necessary in the following areas: user studies, use of robots for telepresence surgical procedures, enhanced system reality, improved system architecture, and improved functionality. Table 11 summarizes research needs in each of these areas.

User Studies

There is need to do research among physicians to find out what their problems are and in what ways telepresence and other forms of virtual environments can help them perform their jobs better [102]. Because VE enhances the interaction between users and systems, investigation into "how people interact with the systems ... and how they [systems] can best be employed for instruction, training, assessment, rehabilitation and other people oriented applications" are being undertaken as part of the VREPAR Project [103].

"In terms of future research, it is necessary to upgrade the system by improving not only [virtual reality] related parts such as developing a high resolution [head-mounted display] or the enrichment of the [virtual reality] experience but also other parts such as a vibration pattern or massage contents. Also, it is necessary to evaluate physiological effects of [virtual reality] itself or to investigate more about the physiological meaning of [virtual reality] stimulations to the human" [104].

Telepresence Surgical Procedures

"At MIT, Ian Hunter is developing VE technologies for teleoperated surgical procedures. The initial application uses a teleoperated microsurgical robot for eye surgery. A virtual model of the eye has been developed to practice teleoperated surgery under simulated conditions" [105]. Hunter is also developing a simulated beating heart that would allow surgery on a beating heart [106].

Enhanced System Reality

While inclusion of the visual, aural, and tactile senses into virtual reality systems is widespread, the sense of smell has been largely ignored. A mechanism for transmitting olfactory information for telepresence is needed. "This would have direct applicability in the area of telemedicine since the sense of smell is an important sense to the physician and surgeon" [107].

"The Virtual Perception Program at SRI International has been exploring the use of spoken language for [virtual environment commands with SRI's Decipher, a speaker-independent, continuous-speech recognition system. Spoken language proved very useful for discrete commands, such as selecting virtual objects and changing their attributes, but poor for continuous commands such as navigating through a [virtual environment]. Spoken commands for movement to specific locations, such as a chair, also were useful" [108].

Table 11. Research Areas for Virtual Environments in Health Care

Demand for enhanced-reality "services is being helped by minimally invasive surgery, such as endoscopy, where the doctor guides an optical-fiber light probe and tiny camera through the patient while looking at a video screen. Instruments are inserted through puncture holes and manipulated from outside the body. A 3-D environment to utilize human depth perception might aid this work, as might tactile feedback since often cancerous tissue is detected by touch" [109].

Improved System Architecture

"There have been attempts of 3D visualization in medicine, most of them follow the vector based approaches of the VR world. The main sources of medical 3D data are CT and MR images" [110]. Fast parallel hardware can and will solve the time bottleneck, and within the next few years, virtual environments in medicine will be realized if the segmentation and classification problems can be overcome [111].

Improved Functionality

To achieve improved functionality in virtual environments for health care, the hardware should run faster. The display should be brighter, sharper, and higher-resolution; when coupled with a head-mounted display, it should also be more lightweight and contain faster and more accurate head tracking. The system software should support the faster hardware. Networking should support real time multi-user environments, eventually allowing an infinite number of users to network with real time results (including speech) over, say, a telephone line. The interaction device should provide force feedback and support 3-D navigation, object interaction, flawless speech generation and recognition [112].

Appendix: Medical Diagnosis and Monitoring (Telemedicine)

There is a growing trend toward the development of telemedicine applications: "In 1994, 37 telemedicine projects were launched, joining some 20-odd telemedicine pilots already in progress" [113]. The potential for involvement ranges from what has become traditional telemedicine functions (diagnosis and monitoring) to remote surgery. Table 12 list some specific applications of virtual environments and related technologies for telemedicine being explored in current projects. Clearly the potential for expanded and enhanced telemedicine is great.

One key area of telemedicine applications is for the delivery of trauma care in the field. "For example, the health care system in a battlefield situation demands timely and efficient evaluation, development and implementation of a treatment plan for severely traumatized patients." A delivery system should increase the efficiency of centrally located experienced trauma specialists by assisting medics at the remote site [114].

Also challenging is the delivery of expert medical care in remote areas of the United States, as well as in developing countries. The publisher of the Global Telemedicine Report (AJ Publishing, Germantown, Maryland) recently published a map entitled "1995 Telemedicine Across the U.S." It shows telemedicine hubs in all but two states. Other reports indicate that the missing states have at least remote referral sites.

The use of digital compression to send high-quality video images from rural areas to big-city hospitals for diagnosis has been instrumental in bringing specialized care to rural America. "Packets of information, such as patient records, X-rays, or pathology slides, can be sent over the telecommunications link to be stored digitally for review by a specialist. These data can be simple text, the primary care physician's notes on an initial examination, for example, or contain high-resolution images from CT or MRI scans." In addition to interactive videoconferencing, which allows patient and doctor to see and hear each other, new devices let physicians listen to a patient's heart, take blood pressure readings, and otherwise thoroughly examine a patient without actual physical contact [115]. Once the capability exists to display registered images at the remote site, an expert can guide a remote medic by manipulating the graphics that appear on the screen [116].

As for developing countries, their needs are not being ignored. For example, the Cleveland Clinic Foundation is exploring opportunities for providing expertise abroad. Clearly telemedicine is a developed technology. However, telemedicine applications vary from limited electronic exchange of records for review to sophisticated interactive systems with extensive diagnostic equipment.

A "cost-cutting trend is to treat more patients at home using telemedicine technologies rather than in the hospital. A project by Carnegie Mellon University (CMU), Pittsburgh, has combined wireless data communication and laptop computers to help nurses provide care at patients' homes..,[the system allows] nurses to receive doctors' orders, chart patients' progress, review medical reference books, and even show educational videos to patients" [117]. Such systems allow trained medical personnel to provide home monitoring and also allow self-monitoring. A recent study by Stanley M. Finkelstein of the University of Minnesota illustrates the potential of such a system. In his experimental setup, "lung transplant patients, who benefit from monthly checkups after surgery, test themselves, record the results, and transmit the results reliably from their homes. Not only may costs be cut, but daily tests (of blood pressure and 'best blows' into a spirometer) could help detect trends before symptoms indicate a problem. All the measurements were performed daily, stored automatically, and transmitted weekly to a relational database" [118]. The Medical College of Georgia in Augusta, Ga., is putting interactive video and medical diagnostic equipment in the homes of "revolving-door" hospital patients suffering from chronic conditions. "Dr. Jay Sanders, director of the telemedicine center, who in 1991 founded a telemedicine network that has grown to include 59 Georgia hospitals and clinics claims the network has allowed 86% of the patients who would have been transferred out of a rural hospital for specialized care to stay put" [119]. Another example of home care began with Collins' search for a way to better monitor babies with cord entanglement problems. "He performed a clinical study...in which the patients were given telemetry/fetal monitoring equipment so that they could perform the monitoring themselves, without a nurse." The doctor linked into the woman's computer and studied the baby's heart rate and the mother's contractions [120].

Other specialized telemedicine units are also being developed. For example, "a 'virtual clinic' at the University of Kansas in Kansas City provides diagnoses and treatment for Parkinson's disease patients in rural areas, many of whom might otherwise go untreated" [121]. Another example is the monthly "tumor board" meeting of the Carolina Medical Center in Charlotte, North Carolina, and the Watauga Medical Center in Boone, North Carolina. The team of expert oncologists, pathologists, and radiologists reviews the particularly complex cases at Watauga's cancer center from its specially equipped telemedicine room at the Carolina Medical Center. The facility has a video camera, communications capability for real-time video transmission, and a 60-inch color monitor. The experts are thus able to interact with the audience of 30 surgeons, chemotherapy specialists, and health-support workers in Boone [122].

Another innovative use of telemedicine is in prisons. "East Carolina University [ECU] is performing telemedicine consultations to the largest prison in N.C... . Originally established to provide only emergency consultations for trauma cases, this network's usage has expanded to include 31 School of Medicine physicians from 15 medical disciplines. Physicians see and talk to the patients via the telemedicine link and then diagnose and prescribe medications, when necessary. Practitioners have access to a digital stethoscope, a graphics camera and a miniature, handheld dermatology camera to aid in patient examinations" [123]. The network is being expanded to six rural hospitals and a large

naval hospital. A similar telemedicine prison project at the University of Texas, Medical Branch at Galveston, handles 45-65 telemedicine cases per week [124].

Table 12. Applications for Medical Diagnosis and Monitoring (Telemedicine)

There are a number of pilot projects involving regional telemedicine. The following are some examples:

• In Iowa, the three main medical institutions are linked over the statewide fiber-optic backbone called the Iowa Communications Network. The network has hosted 10 telepathology consultations (using the Roche Image Analysis System), 10 telecardiology sessions, 100 echocardiography cases, and 250 non-invasive vascular imaging transmissions [125].
• The Eastern Montana Telemedicine Network's videoconferencing pilot, has had a 95% success rate in its first year of operation, i.e., rural nurses and doctors successfully treated patients who would have been referred to a specialist [126].
• Cleveland Memorial Hospital in Shelby, North Carolina, operates one of the busiest emergency rooms in the state. Telemedicine is used to improve the operating efficiency of this busy trauma center [127].
• Because of the costs involved, the Oklahoma Telemedicine Network, which will ultimately connect 45 rural hospitals in the state, "has decided not to include any videoconferencing in its initial phase. Instead, the prime focus is on the limited transmission of reports and slides [that the Health Care Financing Administration] will reimburse" [128].
• The High Plains Rural Health Network is a growing network of providers. It covers the "frontier areas" (less than six inhabitants per square mile) of northeastern Colorado, northwest Kansas, and southwest Nebraska [129].

References:

[1] Stephen R. Ellis, What Are Virtual Environments? IEEE Computer Graphics and Applications 14, 1 (January 1994) 17-22.

[2] David R. Pratt, Michael Zyda, and Kristen Kelleher, Virtual Reality: In the Mind of the Beholder, IEEE Computer 28, 7 (July 1995) 17-19.

[3] Richard M. Satava, Virtual Reality Surgical Simulator: The First Steps, Surgical Endoscopy 7 (1993) 203-205, and in:

VR93: Proceedings of the Third Annual Conference on Virtual Reality, London, April 1993. Meckler Ltd., London, 1995, pp. 103-105.

[4] Woodrow Barfield and Claudia Hendrix, Factors Affecting Presence and Performance in Virtual Environments. In:

Interactive Technology and the New Paradigm for Healthcare. IOS Press, Washington D.C., 1995, pp. 21-28.

[5] Satava, 1993.

[6] Larry F. Hodges, Rob Kooper, Thomas C. Meyer, Barbara O. Rothbaum, Dan Opdyke, Johannes J. de Graaff, James S. Willford, and Max M. North, Virtual Environments for Treating the Fear of Heights, IEEE Computer 28, 7 (July 1995) 27-34.

[7] Duane K. Bowman, International Survey: Virtual-Environment Research, IEEE Computer, (June 1995) 56-65.

[8] Werner K. Doyle, Interactive Image-Directed Epilepsy Surgery: Rudimentary Virtual Reality in Neurosurgery. In:

Interactive Technology and the New Paradigm for Healthcare. IOS Press, Washington D.C., 1995, pp. 91-100.

[9] Richard M. Satava, Medicine 2001: The King Is Dead. In:

Interactive Technology and the New Paradigm for Healthcare. IOS Press, Washington D.C., 1995, pp. 334-339.

[10] Bowman, 1995.

[11] Satava, 1995.

[12] Satava, personal communication, August 1995.

[13] Satava, personal communication, August 1995.

[14] Satava, 1992.

[15] Anthony M. DiGioia III, Branislav Jaramaz, Robert V. O'Toole III, David A. Simon, and Takeo Kanade, Medical Robotics and Computer Assisted Surgery in Orthopaedics: An Integrated Approach. In:

Interactive Technology and the New Paradigm for Healthcare. IOS Press, Washington D.C., 1995, pp. 88-90.

[16] S. M. Blumenfeld, Novel Imaging for Novel Therapy. In:

Interactive Technology and the New Paradigm for Healthcare. IOS Press, Washington D.C., 1995, pp. 45-47.

[17] John A. Adam, Technology 1994: Medical Electronics, IEEE Spectrum 31, 1 (January 1994) 70-73.

[18] Satava, 1995.

[19] Bowman, 1995.

[20] Doyle, 1995.

[21] Satava, 1995.

[22] Blumenfeld, 1995.

[23] Satava, 1992.

[24] Satava, 1993.

[25] John C. Goble, Ken Hinckley, Randy Pausch, John W. Snell, and Neal F. Kassell, Two-Handed Spatial Interface Tools for Neurosurgical Planning, IEEE Computer 28, 7 (July 1995) 20-26.

[26] Satava, 1995.

[27] Satava, 1995.

[28] W. Bauer, H.J. Bullinger, A. Hinkenjann, and O. Riedel, Planning of Orthopaedic Surgery in Virtual Environments by the Example of Osteotomy Operations. In:

Interactive Technology and the New Paradigm for Healthcare. IOS Press,Washington D.C., 1995 pp. 29-36.

[29] Bauer et al., 1995.

[30] Bauer et al., 1995.

[31] DiGioia et al., 1995.

[32] Masahiro Kobayashi, Toyomi Fujino, Tsuyoshi Kaneko, Hiroaki Chiyokura, Kanehisa Enomoto, Kimihiko Shiohata, Yuichi Momose, Katsuyuki Kanbe, Katsuhiko Shinozaki, and Naohito Fuku, Virtual Surgery for Fracture of the Mandible. In:

Interactive Technology and the New Paradigm for Healthcare. IOS Press, Washington D.C., 1995, pp. 175-179.

[33] Giuseppe Riva, Virtual Reality and Body Experience: A New Approach to the Treatment of Eating Disorders, The International Journal of Virtual Reality: A Multimedia Puiblication for Professionals 2, 2 (1996) 9-16 and

Giuseppe Riva, The Virtual Environment for Body-Image Modification (VEBIM): Development and Preliminary Evaluation, Presence: Teleoperators and Virtual Environments 6, 1 (February 1997) 106-117.

[34] John P. Wann and Jane D. Turnbull, Motor Skill Learning in Cerebral Palsy: Movement, Action, and Computer-enhanced Therapy, Bailliere's Clinical Neurology 2, 1 (April 1993) 15-28 and

VREPAR Project <http://www.ehto.be/ht_projects/vrepar/stroke.htm>.

[35] VREPAR Project <http://www.ehto.be/ht_projects/vrepar/movement.htm>.

[36] Hodges et al., 1995.

[37] Bowman, 1995.

[38] Dorothy Strickland, A Virtual Reality Application with Autistic Children, Presence: Teleoperators and Virtual Environments 5, 3 (Summer 1996) 319-329.

[39] Angela Buckert-Donelson, Heads Up Products: Virtual Worlds Ease Dental Patients, VR World 3, 3 (May/June 1995) 6.

[40] Hiroshi Oyama, Tatsuo Miyazawa, Masaki Aono, Ryutarou Ohbuchi, and Susumu Suda, VR Medical Support System for Cancer Patients: Cancer Edutainment VR Theater (CEVRT) and Psycho-Onocological VR Therapy (POVRT). In:

Interactive Technology and the New Paradigm for Healthcare. IOS Press, Washington D.C., 1995, pp. 433-438.

[41] Wendi E. Black, Virtual Reality: I'm Pedaling as Fast as I Can, Popular Science 245, 4 (October 1994) 43.

[42] David Kahaner, Japanese Activities in Virtual Reality, IEEE Computer Graphics and Applications 14, 1 (January 1994) 75-78.

[43] Rita Addison, DETOUR: Brain Deconstruction Ahead. In :

Interactive Technology and the New Paradigm for Healthcare. IOS Press, Washington D.C., 1995, pp. 1-3.

[44] Debra A. Lieberman, and Stephen J. Brown, Designing Interactive Video Games for Children's Health Education. In:

Interactive Technology and the New Paradigm for Healthcare. IOS Press, Washington D.C., pp. 201-210.

[45] Satava, 1995.

[46] Satava, 1993.

[47] Satava, 1995.

[48] Satava, 1995.

[49] Bauer et al., 1995.

[50] John P. Brennan and Jais A. Brennan, Clinical Applications of a High-Performance Computing System for Visualizing and Monitoring Human Labor and Birth. In:

Interactive Technology and the New Paradigm for Healthcare. IOS Press, Washington D.C., 1995, pp. 48-52.

[51] Personal communication with Fuad Jubran, M.D., Cleveland Clinic Foundation.

[52] Edward Edelson, Robo Surgeons, Popular Science 246, 4 (April 1995) 62-65, 90.

[53] Adam, 1994.

[54] Ann Lasko-Harvill, Chuck Blanchard, Jaron Lanier, and Dale McGrew, A Fully Immersive Cholecystectomy Simulation. In:

Interactive Technology and the New Paradigm for Healthcare. IOS Press, Washington D.C., 1995, pp. 182-186.

[55] Jonathan R. Merril, Gregory L. Merril, Rakesh Raju, Alan Millman, Dwight Meglan, Glenn M. Preminger, Rita Roy, and Richard Babayan, Photorealistic Interactive Three-Dimensional Graphics in Surgical Simulation. In:

Interactive Technology and the New Paradigm for Healthcare, IOS Press, Washington D.C., 1995, pp. 244-252.

[56] Satava, 1995.

[57] Satava, 1995.

[58] MusculoGraphics' Limb Trauma Silulator. Evanston, IL: MusculoGraphics Inc.

[59] Adam, 1994.

[60] Hiroshi Oyama, Kazuhiro Nomura, Tatsuo Miyazawa, Masaki Aono, Ryutarou Ohbuchi, and Susumu Suda, Surgical Simulation Support System. In:

Interactive Technology and the New Paradigm for Healthcare. IOS Press, Washington D.C., 1995, pp. 439-444.

[61] John S. McDonald, Louis B. Rosenberg, and Don Stredney, Virtual Reality Technology Applied to Anesthesiology. In:

Interactive Technology and the New Paradigm for Healthcare. IOS Press, Washington D.C., 1995, pp. 237-243.

[62] Adrie C. M. Dumay, Triage Simulation in a Virtual Environment. In:

Interactive Technology and the New Paradigm for Healthcare. IOS Press, Washington D.C., 1995, pp. 101-111.

[63] Myron W. Krueger, Olfactory Stimuli in Virtual Reality for Medical Applications. In:

Interactive Technology and the New Paradigm for Healthcare. IOS Press, Washington D.C., 1995, pp. 180-181.

[64] S. Daane, P. Constantinou, and G. Hesselroth, A $100 Surgical Simulator for the IBM PC. In: Interactive Technology and the New Paradigm for Healthcare. IOS Press, Washington D.C., 1995, pp. 79-80.

[65] Doyle, 1995.

[66] McDonald et al., 1995.

[67] Satava, 1995.

[68] Bowman, 1995.

[69] Walter J. Greenleaf, Rehabilitation, Ergonomics, and Disability Solutions Using Virtual Reality Technology. In:

Interactive Technology and the New Paradigm for Healthcare. IOS Press, Washington D.C., 1995, pp. 415-422.

[70] Satava, 1995.

[71] Arthur L. Zwern and Michael R. Clark, Virtual Computer Monitor for Visually Impaired Users. In: Interactive Technology and the New Paradigm for Healthcare. IOS Press, Washington D.C., 1995, pp. 406-414.

[72] Angela Buckert-Donelson, Up and Coming: University Applied Physics Laboratory, VR World 3, 3 (May/June 1995) 11.

[73] Tom Riess and Suzanne Weghorst, Augmented Reality in the Treatment of Parkinson's Disease. In: Interactive Technology and the New Paradigm for Healthcare. IOS Press, Washington D.C., 1995, pp. 298-302.

[74] Gail Dutton, Virtual Reality: Perpetual Pathway, Popular Science 245, 1 (July 1994) 34.

[75] Angela Buckert-Donelson, Heads Up Projects: Supplier of Health Care Equipment Adopts VR, VR World 3, 3 (May/June 1995) 4.

[76] Angela Buckert-Donelson, Heads Up Projects: Disabled Learn to Use Public Transport with VR, VR World 3, 3 (May/June 1995) 4-5.

[77] Bowman, 1995.

[78] Satava, 1995.

[79] Larry Krumenaker, Virtual Reality: Roll-Through Blueprints Popular Science 244, 5 (May 1994) 48.

[80] Buckert-Donelson, 1995.

[81] Chris Chinnock, VR Can Improve your Health, Byte 20, 5 (May 1995) 26.

[82] Paul E. Keller, Richard T. Kouzes, Lars J. Kangas, and Sherif Hashem, Transmission of Olfactory Information for Telemedicine. In:

Interactive Technology and the New Paradigm for Healthcare. IOS Press, Washington D.C., 1995, pp. 168-172 and Krueger, 1995.

[83] Bowman, 1995.

[84] William J. Peine, Dimitrios A. Kontarinis, and Robert D. Howe, A Tactile Sensing and Display System for Surgical Applications. In:

Interactive Technology and the New Paradigm for Healthcare. IOS Press, Washington D.C., 1995, pp. 283-288.

[85] Robert Langreth, Virtual Reality: Head Mounted Distress, Popular Science 245, 5 (November 1994) 49.

[86] Satava, 1993.

[87] Kahaner, 1994.

[88] T.W. Rowley, Problems of Head Mounted Display. In:

IEE Colloquium on 'Real World Visualization -- Virtual World -- Virtual Reality. IEE, London, U.K, 1991, pp. 12/1-4.

[89] Bowman, 1995.

[90] Kahaner, 1994.

[91] Kahaner, 1994.

[92] Adam, 1994.

[93] Bowman, 1995.

[94] Robert Langreth, Virtual Reality: Breakthrough Hand Controller, Popular Science 246, 1 (January 1995) 45.

[95] Bowman, 1995.

[96] Kahaner, 1994.

[97] Keller et al., 1995.

[98] Ingrid Wickelgren, Medicine: Touchy-Feely Robot, Popular Science 247, 1 (July 1995) 21.

[99] Edelson, 1995.

[100] Richard A. Robb and Bruce Cameron, Virtual Reality Assisted Surgery Program. In:

Interactive Technology and the New Paradigm for Healthcare. IOS Press, Washington D.C., 1995, pp. 309-321.

[101] VREPAR Project <http://www/ehto.be/ht_projects/vrepar>

[102] Ginny Wagner, As Grant Comes to a Close, Iowa Ponders Future of DS-3 Telemedicine Network, Healthcare Telecom Report: The Newsletter of Communications and Information Systems Solutions (Germantown, MD) 3,12 (June 5, 1995) 1-4.

[103] VREPAR Project <http://www.ehto.be/ht_projects/vrepar/aims.htm>

[104] Ryoji Nakajima, Junji Nomura, Shogo Fukushima, and Takeo Ojika, A Relax/Refresh System Applying Virtual Reality. In:

Proceedings: 3rd IEEE International Workshop on Robot and Human Communication. IEEE, N.Y., 1994, pp. 145-150.

[105] Bowman, 1995.

[106] Richard M. Satava, personal communication, August 18, 1995.

[107] Keller et al., 1995.

[108] Bowman, 1995.

[109] Adam, 1994.

[110] Christian Roux and Jean Louis Coatrieux, Visualization in Medicine: VIRTUAL Reality or ACTUAL Reality? In:

Proceeding: Visualization '94. IEEE Computer Society Press, Los Alamitos, CA, 1994, pp. 396-399.

[111] Roux and Coatrieux, 1994.

[112] Joanna Alexander, Jeff Close, Janet Galore, and Mark Long, Welcome to the Next Level: VR Goes Mass Market in 1995, VR World 3, 3 (May/June 1995) 17-21.

[113] Deborah Dakins, Telemedicine Newsletter, Miller Freeman Inc. Publisher, San Francisco, as reported by Alice Laplant, A Virtual ER, Forbes ASAP, (June 6, 1995) 48-58.

[114] O'Toole et al., 1995.

[115] Laplant, 1995.

[116] O'Toole et al., 1995.

[117] Adam, 1994.

[118] Adam, 1994.

[119] Laplant, 1995.

[120] Jason Collins, Louisiana Physician Gears Up for Expanded Tele-Obstetrics, Healthcare Telecom Report: The Newsletter of Communications and Information Systems Solutions (Germantown, MD), 3, 8 (April 10, 1995) 1-2.

[121] Laplant, 1995.

[122] Laplant, 1995.

[123] David C. Balch, Telemedicine in Rural North Carolina. In:

Interactive Technology and the New Paradigm for Healthcare. IOS Press, Washington D.C., 1995, pp. 15-20.

[124] Robert Brecht, Show Report: Telemedicine 2000 Highlights the State of the Art in Telemedicine, Healthcare Telecom Report: The Newsletter of Communications and Information Systems Solutions (Germantown, MD), 3, 12 (June 5, 1995) 8.

[125] Wagner, 1995.

[126] Laplant, 1995.

[127] Laplant, 1995.

[128] Laplant, 1995.

[129] Steven Mecklenburg, Collaborative Medicine Comes to the High Plains via New Network, Healthcare Telecom Report: The Newsletter of Communications and Information Systems Solutions (Germantown, MD), 3, 8 (April 10, 1995) 3.