VR Simulator Sickness Studies
UX/HF Research
A series of studies that investigate the cause, prevalence, and potential interventions for simulator sickness in an industrial vehicle VR simulator.
Read all about it!
Read all about it!
Wait! Need some more background?
Read my latest article on Medium entitled, “Measuring simulator sickness in VR: a guide for UX researchers” which was featured as an editor’s pick on UX Collective and curated as one of the top UXR articles of the week by User Weekly.
Background
Simulator sickness (aka visually-induced motion sickness) is a well-known problem in VR systems, especially for driving-type simulations where virtual movement does not translate to real-life tactile feedback. Because some aspects of simulator sickness are biological in nature or highly sensitive to situational changes, full alleviation is not a practical goal for all individuals. However, there are a number of promising studies citing new methodologies of mitigating these effects which are explored in the studies below.
In this work, I developed a biological proxy for quantifying simulator sickness, identified successful comfort interventions, and discovered key areas of UX improvements in our current simulation software.
Impact
The results of this work have provided insights on how to guide the trainers using our product to successfully complete training with their operators.
The most prudent proxies for biologically sensing signs of simulator sickness have been identified to simplify future usability tests
Trainers can utilize hardware interventions (e.g., electrostimulation bracelets) to prevent simulation sickness during VR simulation or instantly alleviate symptoms after a session.
Our “Train the Trainer” program for VR has been changed to include a section on how to talk about simulator sickness (or more specifically, how to avoid a placebo effect by priming subjects properly).
By identifying key areas of where Unity physics requires modification, the most egregious simulator sickness experiences can be prevented.
Defining Biological Metrics as a Simulator Sickness Proxy
Key Metrics Development, Secondary Research
Background
While it’s incredibly important to notice the signs of simulator sickness in your VR users, quantifying its impact is what will help you solve the following types of questions:
Is everyone experiencing this? To what degree are we seeing symptoms? How do these symptoms impact simulator use? What about its effects on users after they finish their session? Does this experience deter them from returning? How do we know a particular intervention is working?
However, in order to do that, we first needed to formulate a reliable way to determine whether or not subjects were feeling simulator sickness and, if so, how to quantify the severity.
Methods of Measure
-
Subjective Measures
The premiere method of qualitatively assessing simulator sickness is the Simulator Sickness Questionnaire (Kennedy et. al, 1993), which builds off of a previous paper on a Motion Sickness Questionnaire (Kellog, et. al, 1965) in order to adapt it to simulator-specific applications. Administering this questionnaire requires great care to avoid misleading baseline measurements and can only be performed before or after a session.
Behavioral observation and interviews are recommended for attaining additional qualitative information to support these findings.
-
Objective Measures
Due to their overlapping symptomology, previous measures of motion sickness have been evaluated for their utility in simulator sickness (Mazloumi-Gavgani et. al, 2018). Of these measures, a select few can be directly measured with biosensors.
Studies have shown that some of these potential candidates for a simulator sickness proxy may include temperature, skin conductance, and heart rate. Although these measures are objective, they do require extra care in proper sensor placement to accomodate for certain physiological differences between individuals.
Why use proxy measurements?
A proxy is an indirect measure of a desired output which is strongly correlated to that output, commonly used when direct measures of that outcome are unavailable or near-impossible to measure. My reasoning for defining a proxy for simulator sickness is as follows:
Simulator sickness is best observed covertly in order to prevent a placebo effect.
We’ve found during field studies that participants that discuss the possibility of simulator sickness beforehand were more likely to report experiencing it than those who were unaware of the possibility. This means a questionnaire could interfere with the outcome of the experiment if not administered carefully.Questionnaires can only practically be administered before or after an experience, not during.
If you only want to determine a binary yes/no of whether participants experience simulator sickness or look at general severity of symptoms, a questionnaire is appropriate. But if your goal is to determine when participants experience simulator sickness (e.g., to identify a particular UX trigger within an experience), a continuous measurement where you can look for changes of activity is more practical.Subjective measurements can be unreliable and slow to collect.
When it comes to describing how we feel, people can make their own interpretations and provide inconsistent responses, especially for new sensations that they may have trouble describing with words. Therefore, I believe it is worthwhile to try finding an alternative measure that could potentially eliminate these discrepancies and—at some point in the future—automatically detect and correct for problems in real-time with AI.
Experimental Procedure
This experiment took place in our HQ’s VR Lab. Participants were sourced from internal employees including interns, full-time engineers, and forklift drivers from the factory.
Participants were scheduled for an individual session where appropriate biosensors were attached to them as they completed specified lessons in the VR simulator. The full session was recorded with screen capture and video. After the lessons were completed, the participant was given time to rest and fill out the Simulator Sickness Questionnaire and relevant demographic information.
Results
Two key metrics were found to be the most correlated with self-reports of simulator sickness compared to other measures. Further simulator sickness usability testing is now being conducted with these sensors as a proxy measurement.
Alleviating VR Simulator Sickness through Electrostimulation
Experimental Design
Background
Electrostimulation is one new area of research being investigated for alleviating both motion sickness and simulator sickness alike by blocking the nerve signals that create a feeling of nausea, in particular, an upset stomach.
For the VR Forklift Simulator project, we asked our implementation specialist to bring along various potential remedies to experiment with customers anecdotally, of which an electrostimulation bracelet seemed to have a notable effect. This study is a formalized version of those experiments.
User Groups
Our users for this study include two important groups:
Beginner Operators who have never driven a forklift truck before and are using the VR Simulator to learn the ropes for the first time.
Experienced Operators who have driven a forklift truck before, for any amount of time, and are using the VR Simulator to retrain or certify.
The primary reason that these groups must be separated is because the experienced operators group has experience on real vehicles, meaning they expect to feel the force and acceleration of the vehicle while driving, whereas beginner drivers are shown to have no such expectation. This expectation mismatch is the primary cause of why simulator sickness is more prevalent in experienced operators.
Experimental Design
The purpose of the experiment is to test if an electrostimulation bracelet would prevent simulator sickness in subjects. In order to test this, each user group was randomly split into control (no bracelet) and test (with bracelet) groups, which meant we ultimately end up with 4 unique groups in this study.
This experimental design was initially tested with a target minimum sample size of n = 5 for each group to test the experimental design. The next phase of this study—to expand the number of participants to a statistically significant sample size for quantitative analysis—is currently underway.
Experimental Procedure
This experiment took place in our HQ’s VR Lab. Participants were sourced from internal employees including interns, full-time engineers, and forklift drivers from the factory.
Participants were scheduled for an individual session where appropriate biosensors were attached to them as they completed specified lessons in the VR simulator. The full session was recorded with screen capture and video. After the lessons were completed, the participant was given time to rest and fill out a survey and relevant demographic information.
Results
The initial results of this study are promising, showing a reduced tendency to experience simulator sickness when using the bracelet at the start of a VR session for both user groups. It was also identified that experienced operators are more likely to experience simulator sickness of any group, which suggests a strong need to find ways to tailor the experience to each of these two distinct user groups.
Identifying Potential Unity Physics Modifications for Alleviation of Simulator Sickness
Ethnographic Observation, User Interviews
Background
While many generalized software fixes for alleviating simulator sickness have been identified and implemented by our product team in the past (e.g., edge dithering, foveated renfering) one of the most difficult problems to solve is aligning Unity physics to reality in a game environment that is intended to mimic real motion.
Part of this is a Unity developer problem; physics must be explicitly defined in the program for each type of object and are not a given.
Part of this is also an engineering problem; doing the proper experiments in the lab to produce the results needed for those physics modifications is tricky to schedule and measure.
Not only is getting these people together difficult (remote developer team) but implementing and testing solutions is tedious and time consuming, meaning it is expensive for the company to fix every detail.
As a result, this study was focused on identifying the most egregious examples of where Unity physics errors led to significant and/or frequent simulator sickness symptoms.
Methodology
In order to initially identify these problems, I needed to go out into the field and see what was actually happening in real time rather than relying on anecdotes and paraphrased statements from field engineers.
For this, I tagged along with our product manager to customer implementations to take notes on what I could observe about operator and trainer behavior alike.
First, I took note of how trainers interacted with operators. Did they ask operators if they typically experience motion sickness? Were other operators chatting about “feeling sick”?
Second, I took detailed notes of how people fared during actual training on the VR Simulator, paying attention to any cues of simulator sickness and what type of motion was being executed in the simulator at the time. After each operator was finished, I recorded minimal demographic information such as whether or not they are experienced operators.
Insights
Through observing and interviewing subjects, we were able to directly ask experts who drive the real vehicles everyday to discover what felt the most “off” to them and where simulator sickness symptoms were most strongly triggered.
Here we found two major contributors to simulator sickness: inaccurate acceleration curves caused the virtual vehicle to decelerate too quickly and turns were too sharp. From there, we were able to work with the relevant engineers to gather and test the correct values to replace in Unity, which was then retested to validate this feature change for live release.
This endeavor has cut simulator sickness complaints in experienced operators down significantly since this fix was implemented.
That's all for now!
That's all for now!
Eager for more?
Click the button below to check out some other projects!