Mixed reality (MR), also referred to as extended reality (XR), combines the physical and digital realms and encompasses a number of technologies—including augmented reality (AR), virtual reality (VR), 360-degree video, and holograms.

Each of these technologies requires an interface of some kind: touch or gesture-based, voice-controlled, an eye tracking system, or a combination of various bio-inputs. Nearly every industry can use these technologies, whether it’s teaching anatomy to medical students and training remote employees in technical skills, to experiencing virtual prototypes of cars and attending virtual concerts of artists (Tupac, Elvis) who are long dead.

A prominent device in the mixed reality space, Microsoft’s Hololens 2 relies on eyelined-based user interactions. At the beginning of each use, the headset is calibrated to the unique shape and movement of a user’s eye. Eye tracking enables applications to track where the user is looking in real time.

For example, following a user’s gaze helps the system understand intent. Combined with a voice command, a user can look at a desk and say “put this here” without having to specifically name or describe the desk. There are implicit use cases, too: If someone is reading a screen with a lot of text, the page can automatically scroll to keep pace with her eyes. As mixed reality developers aim to reproduce the nuances of human vision, including depth-of-field, this eye-tracking technology could also prove quite useful in establishing the user’s point of focus.

The Food and Drug Administration has approved multiple surgical planning apps for Microsoft’s Hololens mixed reality headset. Florida-based Magic Leap is expanding its developer program for the Magic Leap One MR headset, which will mean new consumer applications and games in the coming year.