Motion-Coupled Virtual Environment (MOCOVE)

Final Report for Phase I Effort Supporting Topic N99-183

10 June 2000

Prepared For:

Naval Air Warfare Center Training Systems Division

12350 Research Parkway

Orlando, Florida 32826-3224

Technical Abstract

The team of ARTIS, LLC and Brandeis University addressed three objectives in Phase I. First, we developed a system, the Motion Coupled Virtual Environment (MOCOVE), capable of coupling the sensed motion in the environment to the visually perceived motion in the VE. Second, we tested MOCOVE in the laboratory to determine its capabilities. Third, we designed an experiment involving human subjects within the MOCOVE deployed on a laboratory moving base, whose main objective will be to guide the development of goals for our Phase II plan. All objectives were achieved. This research will lead to a better understanding on the use of motion-coupled environments in reducing side effects associated with virtual environments as used on moving platforms such as aircraft and ships. Potential commercial applications include military training systems that exhibit fewer side effects and novel, commercial entertainment systems.

Key Words

virtual environment cybersickness

human computer interface simulator sickness

military training entertainment systems

motion sickness ship systems

 

 

 

 

 

ARTIS, LLC Dr. Jed Marti (Artis, LLC), Principal Investigator

1914 Foxhall Road Mr. Keith Brendley (Artis, LLC)

McLean, Virginia 22101-5535 Dr. Paul DiZio (Brandeis University)

www.artisllc.com Dr. James R. Lackner (Brandeis University)

1. Overview

The advent of Virtual Environment (VE) technology presents opportunities for creating more flexible and effective training systems with naval applications. While not yet fully established, VE training systems hold forth the promise of reduced training costs, minimal risks to personnel and equipment, and a higher state of force readiness. Furthermore, as the U.S. Navy considers approaches for reducing crew manning requirements, VE trainers may serve as a critical enabler for maintaining broad skill sets.

To fully realize their potential, VE training systems must be used aboard ships and submarines while they are at sea and in harbor. A key challenge in this environment is to reduce the side effects of VE, most notably motion sickness, postural instability and spatial disorientation. Taken as a whole, these side effects are sometimes referred to as "cybersickness."

The discontinuity between actual motion and perceived motion in the VE has been shown to be among the greatest contributing factors. Research from members of our team [DiZio and Lackner, 1998] has shown that more severe effects may be induced by increasing the lag time between VE scene update and expected update due to head motion. Of course, a ship at sea moves independently from any VE that may be on board, and if cybersickness is the most recently recognized form of motion sickness, then sea sickness is one of the oldest (Reason & Brand, 1975). For ship-board use, then, trainees will be exposed to ship motion that is discordant from that perceived in the VE, a "motion discordant environment."

To reduce the anticipated negative effects exposure to a motion-discordant environment, we developed a research program and explored the base technologies that will result in a motion-coupled training system to significantly reduce the inducement of cybersickness.

ARTIS, LLC teamed with Brandeis University to create Team ARTIS, which is fully capable of developing the hardware and software required for a motion coupled VE, studying its efficacy, and developing a system that meets U.S. Navy training needs.

This report is divided into six sections, including this overview. Section 2 describes the research conducted in Phase I. Section 3 describes the test stand used to calibrate MOCOVE sensing systems. Section 4 describes the sensing system itself, including details of hardware and software and results from calibration tests. Section 5 presents the simulation system used to create the virtual environment that will be used for human experiments in the Option Phase and initial portions of Phase II. Section 6 discusses the experimental plan that will be used in the Option Phase. Finally, Section 7 presents the commercialization strategy that will be applied in later phases of the effort.

Our findings are as follows:

  1. Motion-discordant environments impose serious side effects on users, implying potential problems for use of virtual environments on motion platforms such as ships and aircraft

  2. We designed and built the first prototype of a low-cost sensing system that exhibits the necessary qualities to enable the development of a motion-coupled virtual environment.

  3. We developed the necessary systems and experimental protocol to conduct meaningful human experiments in the Option and later phases of the planned course of work.

  4. By alleviating side-effects of motion-discordant environments, MOCOVE is a key enabling technology that will permit Artis, LLC to pursue significant licensing agreements with training system providers.

    5.  

     

    2. Research Conducted

    This Phase I SBIR conducted a multi-phased research plan resulting in an inertial measurement device currently being connected to the Brandeis Vertical Linear Oscillator (VLO). The following important developments were made:

  5. A vertical pitch and heave test stand. This small device permits pitch (30 degrees) with .2 degree accuracy and heave (1 meter) measurements at the ARTIS laboratory.

  6. The MOCOVE prototype tested both gyroscopes and MEMS accelerometers.

  7. The hardened prototype circuit board was specifically designed for MOCOVE and includes two, 2 axis accelerometers and better power conditioning.

  8. A ship board simulation written in Open GL was connected to the MOCOVE prototype on the test-stand.

  9. The hardened MOCOVE prototype has been mounted on the Brandeis VLO and data is being taken.

    10. 3. The MOCOVE Test Stand

    We first constructed a motor-drive test stand to simulate a ship’s pitching and heaving. The support stand is constructed out of 3/4" plywood and consists of a base and stand for the drive motor and a tower to support the experiment beam and it’s fulcrum.

    Pitch and heave test stand.

    The 1/50th horsepower geared drive motor is sufficiently powerful to swing the experiment area through about 3’ and about 30 degrees at a maximum speed of 1/20th of a hertz. A controller allows us to control both its direction and speed. The motor is attached to a one foot diameter driving wheel. A drive rod connects the wheel to the experiment beam.

    The experiment beam is a 6’ long extruded aluminum beam with bearings at the pivot point. We added a lead counterweight during later experiments to minimize mechanical vibration and offset the weight of the constant angle device.

    The pivot shaft is drilled and tapped to attach to the 1/4" shaft of the angle measurement potentiometer. It’s purpose is to provide an accurate, real-time measurement of the beam’s angle against which to compare the output of the MOCOVE device.

    Fulcrum and potentiometer angle measurement.

    We tested a number of potentiometers for linearity and to minimize the test-stand cost. The selected potentiometer is connected to a low-cost, 8 bit analog to digital converter based on ARTIS’ MINES technology. The device communicates with the SGI workstation over a 2400 baud serial link. The device can be either battery powered or powered by a remote "wall wart" 12 VDC power supply.

    MINES device for measuring support beam angle.

    Initial experiments computed the device angle from measurements and calculations. This was improved with some direct measurements using a level and ruler. By these measurements, the test stand varies the device in pitch from -9.9 degrees to 19.9 degrees and with an accuracy of somewhat better than .25 degrees.

    The MINES device calibrates itself during each test run (the potentiometer’s resistance varies with ambient temperature). It collects the minimum and maximum resistance values and scales this appropriately for the measured angles. Once a complete up and down cycle is complete, it begins sending angle measurements in tenths of a degree at 2400 baud. To avoid communication problems, each measurement is sent with synchronization and checksum bytes. Examination shows that it can discern changes down to about .2 degrees.

    4. The MOCOVE Device

    The MOCOVE device has evolved into a small, battery powered, micro-controller using two, 2G, 2 axis accelerometers. It samples these at about 15 hertz and communicates the values through a 19,200 baud serial port.

    4.1. Design and Construction

    The design and construction went through several phases from breadboard to a reasonably ruggedized prototype.

    4.1.1 Prototype and MINES

    The first tests were completed on a bread board device shown below. The processor board features a PIC 16C73 8 bit micro-controller, 4k serial EEPROM, RS232 communications, a 16x2 LCD for debugging and connections to a single accelerometer and gyroscope.

    MINES breadboard of MOCOVE device.

    Testing ensured that both devices could be read and that the values returned were reasonable. Both devices require special mounting both to position them correctly and to isolate them from shock, low frequency mechanical noise, and electrical noise.

    Prototype mounting of accelerometer and gyroscope.

    Coding during this period centered on communicating with the accelerometers. These send variable width pulses depending on the force of gravity along a vector. A precision resistor sets the pulse width that the micro-controller measures to within about 2 microseconds. The current setup has a duty cycle of about 10 milliseconds — it takes up to 80 milliseconds to acquire all 4 values. The duty cycle can be set to 1 millisecond, but with loss of accuracy.

    4.1.2 Prototype for test stand

    Assured of the Moscow’s device’s proper operation, we then constructed a battery-powered box to mount on the test stand. Using the same prototyping board, we mounted a single accelerometer and gyroscope and 8x2 LCD inside a plastic box. The device is now powered by 6 AA batteries for operation over several hours (the gyroscopes consume considerable power).

    MOCOVE demonstration prototype ready for mounting.

    This prototype was used in the initial gyroscope and accelerometer test stand operation. It was mounted on the end of the test stand beam and its communication cable run to the Silicon Graphics O2. Continuous operation caused the test stand drive motor to overheat — this was corrected by bolting a lead counterweight to the support beam.

    MOCOVE demonstration prototype mounted on support beam.

    The prototype’s cable and construction were not strong enough for use on the VLO and repeated wire tweaking eventually destroyed its electrical integrity. The device did survive long enough for the demonstration on April 26.

    4.1.3 Hardened Prototype

    Testing revealed that the rate gyroscopes were not sensitive enough to the low angle rate changes envisioned for the device. Consequently, the gyroscope was removed and replaced with an additional accelerometer. A printed circuit board based on the prototype was designed and fabricated. It contains the micro-controller board with RS232 level converters, power conditioning, serial EEPROM and DB9 serial interface connector. Two additional boards cut from the master contain the surface mount accelerometer chips, their power conditioning and timing resistor.

    MOCOVE hardened prototype with single accelerometer.

    The first box proved to be too tight a fit for both the 9v battery and main circuit board. Subsequent boxes are slightly larger with more room for power conditioning, LED display and wiring.

    4.1.4 Software and Electrical Improvements

    Tests showed that the accelerometers were greatly affected by electrical noise coming from the microcontroller. Various attempts to palliate this were unsuccessful until both units were isolated by separate voltage regulators. An add-on regulator board has been designed and will be incorporated into subsequent units.

    Accelerometer noise — device motionless.

    After correction of angle, comparison to the analog measurement, the noise is severe.

    Accelerometer noise — device in motion.

    We tried a software filter to smooth the results. The following graph shows smoothing results for various levels of filtering. As you can see, the higher levels start to introduce phase shifting — an undesirable result.

    Detail results of digital filter on raw data.

    We tried various mechanical means to remove noise — the motor drive vibrates the support beam to a certain extent — all without success. We then tried electrical means to filter power to the accelerometers all without much effect. Finally, we supplied separate power to the accelerometers through their own regulators attached to the battery before the micro-controller’s regulator. The results are much improved — unprocessed accelerometer output is shown in the following graph in red, with unfiltered values in black.

    Regulated power used to drive accelerometer.

    4.2 Testing and Calibration

    The MOCOVE device is positioned at several measured angles and multiple value multiple X and Y accelerometer values and angle values are written to a file. The measurements are only taken while the device is not moving to remove effects of radial velocity and cyclic accelerations. The device was stopped at 7 different angles as measured by the angle potentiometer and a level (for zero degrees). Approximately 80 readings were taken at each station and averaged.

    We then computed an un-normalized angle f as above and computed a linear least squares fit to the measured angle as shown in the following graph:

    Computing the angle from raw accelerometer values.

    This suffices for pitch measurement with low radial velocities. However, a more complex analysis is required for pitch and heave.

    4.3 Computing Pitch and Heave

    The accelerometer can be considered as a set of two orthogonal vectors.

    Let the normalized accelerometer readings be Vx and Vy. The accelerometer is modeled as a pair of vectors and . We wish to compute the vector .

    (1)

    (2)

    (3)

    Make the accelerometer vectors unity (). Then:

    (4)

    (5)

    (6)

    Then, is computed by back substituting into which ever equation has a non-zero denominator.

     

    (7)

     

    The Y component is integrated over the time step to derive the heave position. Since the accelerometer reads 1.0 (1 gravity) when pointed away from the earth’s center, we have.

    (8)

    During phase II we will incorporate this model into the simulation for both pitch and heave. Similar analysis will be performed for the roll vector.

     

    5. The MOCOVE Simulation

    We implemented a simple real-time, naval simulation to demonstrate the MOCOVE device in the real world coupled to a virtual environment. We designed the simulation to place the user on a virtual ship that is pitching and heaving with heaving directly simulated by movement of the VLO. To encourage immersion, additional nearby objects pitch, roll and heave at different rates. A fixed horizon has some simulated terrain to provide a stable reference point. To provide additional realism, the viewpoint and ship moves on a predetermined path under control of a scenario description file.

    Simulation Organization.

    In the following figure we see the simulation shortly after startup. As a viewer, you are standing on the USS Theodore Roosevelt, CVN 71, looking forward. You are standing on the edge of the landing strip looking towards the forward catapults. In the distance is the green shore line with a tall green tower (where 0 east, 0 north is). On the port side you are just beginning to see a simulated battleship that is pitching, heaving, and rolling at a rate slightly different than the ship you are on. To starboard is a small fishing trawler pitching, rolling, and heaving at an alarming rate. The initial scenario also features a lane of red and green buoys and other ships both near and far. Multiple instances of these ships can be created in the scenario file without modifying the program. Their pitch, heave, and roll rates, amplitudes, and phase can be set.

    Typical simulation view.

    For the first demonstration, the simulation’s viewpoint pitch was set by the prototype MOCOVE device with a single accelerometer. The MOCOVE device communicates with the Silicon Graphics workstation with RS 232 communications at 19,200 baud. The first prototype has an LCD for debugging purposes. The device is mounted on ARTIS’ pitch and heave test-stand either at the fulcrum or at the end of the support arm as shown earlier.

    6. Experimental Plan — Using the MOCOVE device

    Systematic observations are required to compare the side effects and aftereffects of simulated ship motion using: 1) just the VLO (vertical oscillation while viewing a real environment), 2) an optimized MOCOVE (virtual visual ship motion matching as perfectly as possible the real vertical oscillation, 3) a VE not coupled to the motion of the VLO (no visual scene motion during real oscillation) , and 4) various forms of degraded MOCOVE (virtual visual motion partially matching the real oscillation, for example correct phase but reduced gain). A small set of experienced observers will report their motion sickness, disorientation and postural side effects and aftereffects.

    These observations will help define full experiments for a Phase II proposal that will 1) define areas in which the MOCOVE hardware and software should be improved, 2) define system and user characteristics that will circumvent side effects and aftereffects, 3) optimize acceptance and effectiveness of MOCOVE training systems

    Research announcement N99-183 established the objective of developing "strategies and methods for linking real-world motion with perception in a virtual environment." The Navy desires to deploy virtual environments in various transportation systems, creating the possibility of presenting the user a visual scene whose motion that is not linked to the real motion that the user is exposed to. This is one of the prime factors inducing motion sickness (DiZio & Lackner, 1992, Kennedy et al, 1997). In response, Artis, LLC has developed motion coupled virtual environment (MOCOVE) hardware and software. The prototype system is capable of inertial tracking of the motion of a platform and updating the simulated visual perspective of an observer aboard a naval surface ship having the same motion profile as the motion base. The hope is that MOCOVE will circumvent side effects and aftereffects. This system has been delivered to the Ashton Graybiel Spatial Orientation Laboratory of Brandeis University for evaluation of human performance.

    The MOCOVE system has been integrated with the Brandeis vertical linear oscillator (VLO) for the purpose of evaluating what factors potentially affect human performance. The following figure shows the MOCOVE tracking device on the carriage of the VLO.

    Left: Photograph of the VLO with the MOCOVE sensor mounted on the carriage. Insert at right: detail of the sensor.

    The VLO has been chosen as a test bed because previous work at Brandeis has shown the following:

  10. Motion sickness, postural instability and manual performance errors are serious side effects of 15 minute exposure to VEs with sufficiently poor motion coupling. The 15 minute exposure paradigm, with side effect measurements at 3 minute intervals, has repeatedly been validated as a reliable way of comparing side effects across different VE systems and exposure conditions.

  11. Viewing a real environment while oscillating on the VLO at .2 Hz, as low as 1.2 m peak to peak elicits motion sickness and postural side effects within 20 minutes. In this respect, it adequately simulates sea sickness during exposure to real ship motion (Lawther & Griffin, 1986). Thus, it has operational relevance to the Navy and Marines.

  12. Combined exposure to the VLO and to completely uncorrelated virtual visual motion exacerbates side effects and after effects. None of our previous research has attempted to measure the consequences of linking the virtual visual motion to the real motion of the VLO. The purpose of the first six months of this project was to provide a MOCOVE with the best possible correlation of real and virtual motion, in order to make such testing possible..15

    13. At the April 26th meeting in Salt Lake City of representatives from NAWCTSD, Artis and Brandeis, the MOCOVE was demonstrated on a desktop silicon Graphics computer, tracking the motion of a simple oscillating jig. As the above figure documents, the MOCOVE system has now been integrated with the Brandeis VLO and Silicon Graphics computer. The virtual environment can be experienced by a user wearing an n-Vision datavisor 10x VGA helmet mounted visual display. Formal data are being collected to evaluate the performance tracking performance of the MOCVE sensor.

    The Brandies investigators have determined that the most effective way to use the three month option period of this contract is to answer the question, "How effective is the MOCOVE system, with linked real and virtual motion, at preventing side effects relative to 1) an uncoupled VE, 2) a partially coupled VE and 3) no VE (oscillation in a real environment)." We will try out as many partially coupled scenarios as possible. For example we can easily alter the system to keep the phase of visual-inertial motion linked but to reduce the gain. It is also easy to provide visual linkage only above a certain threshold of inertial motion. Our survey of "partial linkage" conditions will be designed to find the minimal linkage required to ameliorate side effects. An outline of the experimental conditions is shown in the following table:

    15-minute Susceptibility Test (with head movements)

    15 minute Rest

    2 minute Sensitization Test (with head movements)

    VLO eyes open

    Eyes open Sit quietly in a stationary environment

    VLO + MOCOVE optimized

    VLO + MOCOVE optimized

    VLO + VE non-coupled

    VLO + MOCOVE degraded

    The procedure will be to use our paradigm in which the subject makes head movements in the VE at a controlled pace for two minutes followed by a one minute rest period in which side effect measurements and other observations are recorded. This is repeated five times for a total of fifteen minutes exposure. The updating of the virtual visual scene perspective compensating for head movements will be accomplished by an existing tracking system that introduces the minimal possible delays. Thus the VE task per se will elicit minimal symptoms, allowing us to focus on the effects of motion coupling. The VLO will be run at 0.2 Hz, 1.9 m peak-to-peak amplitude because we have a large database at these oscillation settings. The subjects will be six laboratory members who have extensive experience with VEs, motion sickness ratings, and other aspects of spatial orientation research. They will be exposed to all four experimental conditions on different occasions. Observations they will record include the following:

  13. Static distortions of the scene due to tracking of the VLO. Subjects will make estimates of visually perceived tilt of the environment and apparent visual drift of the scene up or down.

  14. Apparent realism of the VE. Subjects will rate their sense of presence in the VE.

  15. Spatial orientation in the VE. After visually scanning the VE subjects will attempt the orient themselves to objects by facing them with eyes closed.

  16. Motion sickness. Signs and symptoms of motion sickness will be rated using the same scale we have been using in previous research.

  17. Postural disequilibrium. Subjects will be seated during experimental sessions, but they will provide verbal reports of postural instability immediately after leaving the MOCOVE.

Subjective reports in all of the categories described above will be compiled and compared across conditions to determine whether optimal MOCOVE is an improvement over the other three conditions. The presence of side effects in a specific measurement category will focus our Phase II objectives upon 1) what human experiments are required to establish tracker performance specifications that will circumvent the side effects and 2) making the requisite technical improvements in MOCOVE. If the MOCOVE performs so well that no problems are reported, then we will pursue accelerated development of the system in Phase II and define human factor tests under expanded, more realistic motion conditions.

 

7. Commercialization Strategy

ARTIS, LLC specializes in developing Information Technology systems and specialty HCI equipment and virtual environments. Our technical staff is comprised of highly trained engineers and scientists, all of whom possess higher degrees. Our two-pronged strategy in Phase III capitalizes on our strengths without diluting our resources in wholly new fields.

First, we will meet low production-level needs for specialty, motion-coupled training systems. Our approach here would be to identify specific training systems that would benefit from the inclusion of a motion-coupling system and to secure sponsors for developing such systems under contract. Second, we would license the technology for higher production-level training systems, and we would provide consulting services for product improvements and other specialty needs.

We view this project as an important opportunity in allowing ARTIS to expand its capabilities in the areas of HCI and VE in a controllable and rational manner. Given the current ARTIS business plan of providing specialty hardware, software and consulting services related to advanced HCI and VE technologies, developing early-entry expertise in a new field such as motion-coupled environments provides us with a significant advantage.

In the area of commercial products, many new devices may be possible. For example, in developing this proposal, we have discussed the possibility of "pocket roller-coasters." Research in the field of motion simulators would indicate that one may impart the feeling of large motion within a VE using only minimal actual motion. One may imagine a relatively small and low-cost roller-coaster coupled with a shared VE. By greatly exaggerating the motion of the actual coaster in the VE, one may be able to deliver high-value entertainment rides in very limited space. In a similar vein, one may also imagine a "virtual bridge" that would be installed on cruise ships where, again, real world motion is exaggerated via a VE to create a more entertaining environment. Clearly, these and other ideas that we have formulated are only concepts at this time, but we believe that this project would enable us to more fully develop and market our capabilities as advanced HCE and VE developers within the commercial market.