Physics and Frame Rate
Many will be aware that one of the big technical risks in Virtual reality projects is motion sickness, or a subset of motion sickness called simulator sickness. Simulator sickness is experienced by people in virtual reality and is well known in pilot training in flight simulators. It has been believed that discrepancies between the motion of the simulator and the subject causes the brain to rebel and even induce nausea.
In our wheelchair trainer this form of motion sickness can compromise training by making the subject unwilling or unable to continue.
We developed the virtual reality wheelchair trainer in May this year. It was a joint project between Callaghan Innovation, Burwood Academy of Independent Living at Burwood Spinal Hospital, and Stickmen Media, our video game development company. MTech games was formed to commercialise the product. The idea was to produce a virtual reality environment for training and assessing people with traumatic spinal injury about using powered wheelchairs.
The result was a success, with a pilot study using current wheelchair users and clinicians, all agreeing that the trainer was just like being in a powered wheelchair. Unfortunately many people reported feelings of nausea after using the trainer. In fact, one collaborator joked that we should establish a bucket list, a list of those subjects who required a bucket during training. As part of the commercialisation effort we had to eliminate simulator sickness.
We began with a list of effects reported by users in the initial pilot study, that contributed to their feelings of discomfort:
- Flicker of the environment, a flickering of the whole environment that happened in particular circumstances;
- Flicker in the models, small areas of the model that flickered persistently;
- A feeling of sliding on corners; and
- A feeling of lightness of the chair, it started and stopped too fast. .
With some investigation we determined the following reasons for these effects.
Frame Rate Judder
The flicker of the entire environment is a common one in VR using headsets such as Oculus Rift. It has been termed judder, and it occurs when the frame rate drops below 75 fps. This was a challenge. Our 3D models are a very accurate representation of the Burwood TransitioNZ spinal unit. The first thing we did was to ensure that all, non moving parts of the environment model were being treated as static. In other words we made sure that the engine (Unity 5) wasn’t wasting cycles to see if those models had moved.
This made quite a difference, but we couldn’t quite tell how much, so we implemented a frames per second counter that could always be seen from the chair. We found that the framerate was at 75fps while the wheelchair was stationary but was mostly below 60fps while travelling. In fact, we found that each time we accelerated the framerate dropped.
The next thing we did was to change the way we read our controller information, using threaded lookups thereby decoupling the controller from the update loop. This caused the frame rate sit at 75fps consistently, even when driving. There are still some places where it drops to 60fps but we suspect that some occlusion mapping in our environment may fix these niggling problems. In-house testing showed a dramatic reduction in reported simulator sickness with the improved frame rate.
3D Model Overdrawing
There were several places in our environment which flickered annoyingly. Many of these seemed on the ceiling, above doors, typically above our normal line of sight. Such flickering, barely in the field of view, can produce nausea and headaches. At the very least it is an uncomfortable visual distraction. As game developers we recognised what caused this flickering. It occurs whenever there are coincident planes in our 3D world. The games engine, having ascertained that both planes are visible, attempts to draw one then the other, in rapid succession.
We eliminated all of these areas, with some millimeter tweaks to the geometry of the environment. Testing again showed an incremental improvement in motion sickness.
The next two items seemed to be related. We have seen previously that the brain is very good at interpreting the physics of the real world producing a set of expectations based on perceived motion. The physics of our chair was a complicated algorithm accounting for its jockey wheels, contra-rotating drive wheels, and drift reduction on corners. Since we were directly calculating velocity, and assuming that the base was frictionless we had two, slight but perceptible problems with our motion:
- The momentum felt wrong, which our users reported as the chair feeling light; and
- There was still a slight feeling of drift on the corners.
Our calculations had been lovingly crafted from observations of an actual power wheelchair on a floor marked out in units using duct tape. Nonetheless it seemed they were not adequate for the harsh reality of VR. We scrapped our calculations and began a meticulous rebuild using a physics engine. We modelled wheels, suspension, motor torque and speed limiting.
Instantly our rebuilt chair ‘felt’ more like the real thing. Its acceleration and deceleration created a more natural sense of momentum. The subtle rock of the chair when it stops instantly enhances the feeling of braking.
Importantly, it caused a huge reduction in reports of motion sickness. In trials of 10 to 15 minutes less than one in six users report any motion sickness at all, and those that do report it as less than 3/10 where zero is no sickness, and 10 is wanting to vomit.
We have made vast improvements to our wheelchair simulator, particularly in reducing motion sickness. If you find yourself struggling with motion sickness in your game or simulation then try these 3 steps:
- Get the frame rate up to 75 fps consistently across the game;
- Adjust your models to remove flicker from coincident planes, in fact, eliminate all flicker;
- Use good physics simulations for moving the oculus camera.
Keep iterating until you’ve nailed it. We have seen now that there are roller coaster simulations that do not cause motion sickness in VR. This is an encouraging fact.
In the meantime, if you’re having reports of motion sickness , keep some mints beside your VR rig. Offer one to anyone who reports motion sickness. They should feel better in minutes.
A PDF file of this article can be found here.
Image from “The Matrix” , Copyright © 1999, Warner Bros