Eye-tracking and foveated rendering have been tipped as the next big things in VR headsets. These two technologies can together create a highly detailed VR experience while minimizing computing power and improving frame rates. San Francisco based company FOVE have recently released their FOVE 0, which amalgamates these technologies into one headset.
Naturally your eyes only focus the objects you are looking directly at, and anything outside of this area is blurred. The further something is away from your focus point; the more blur is present. This is all done by a tiny spot in the back of your retina called the fovea. It also helps you determine depth in the real world. Foveated rendering is an artificial way of producing the same effect, and is useful for VR as it also reduces the computing power needed to run programs. Applications can hit higher frame rates as the amount of rendering required drops. Higher frame rates and lower computation requirements are extremely important for mobile headsets and games. NVIDIA have a provided a great example of the differences in the video below.
This rendering style relies on accuracy though; up until now we have only had headsets that can track your head movements, not your eyes. The head tracking technology is unable to be transferred to tracking your eyes, as eye movement is a lot more unpredictable than head movement. If the eye movement isn’t tracked perfectly it can make the user disorientated and will ruin the experience. FOVE have been working with eye-tracking technology for close to two years and are now satisfied they have created a suitable headset. The FOVE 0 headset has four tiny infrared LEDs on the inside of each lens, lighting up your eyes so the mounted cameras can track the movement. It then blurs the areas around your eye movement, helping to create a more realistic field of depth and allowing the foveated rendering to be accurate. Users don’t even notice the difference.
Above: The FOVE 0 headset sports four tiny LEDs that allow the cameras to detect your eye movement.
FOVE aren’t the only ones playing with this technology though. A similar company, Eyefluence, have been working on eye-tracking since 2013. They have developed low cost solutions and have recently joined Google, hinting that the next headset Google release will have this technology incorporated. Since Google seem to be focusing on the mobile VR gaming, this technology would be extremely important in setting themselves aside from Samsung’s GearVR and Xiaomi’s Mi VR headsets. Available computing power is a big issue with mobile VR headsets.
Another company, BinaryVR, are taking this technology to the next level by adding mouth tracking to headsets. Their expression capturing camera mounts to existing Oculus Rift and HTC Vive headsets, and picks up a range of nose and mouth movements with low latency. BinaryVR is in developer kit production and will be shipped out this month to developers around the world. It has not been announced when the consumer versions will be available, or whether they will be released alongside future products from Oculus and HTC.
With accurate face tracking and foveated rendering being released this year, it will be interesting to see what is in store for us over the coming years.