My Vision for Extended Reality(XR) Headsets

With Apple recently legitimizing XR headsets by announcing their Vision Pro, I wanted to talk about how I expect the headset space to evolve in the near future. I will go into the categories of Extended Reality (XR), a brief overview of their history, the current XR market, and what I think this technology will look like when adopted by the general consumer.

XR- VR, AR, MR

XR is a broad term that encapsulates all tech that modifies your sense of reality with screens.

Virtual Reality (VR) is when those screens occupy your whole vision. It’s most associated with gaming. Examples of this include this technology include the headsets made by Oculus/Meta, HTC, Valve, and even those $9 plastic headsets you can buy for your phone.

Augmented Reality (AR) is when those screens show you the real world, but the additions made by the screens don’t interact with the real world. The most basic version of AR would be those overlay filters on social media. My introduction to this technology was with the Nintendo 3DS which came with AR cards that you could scan to see Nintendo characters walk around in the real world using your camera and 3DS screen.

Mixed Reality (MR) is when the real world and digital world interact with each other. The most notable examples of this would be Microsoft’s HoloLens or Apple's Vision Pro. This category is usually grouped in with AR.

The borders between VR, AR, and MR are extremely blurred. Meta’s newer VR headsets don’t require controllers because they can track your hands which are part of the real world and have cameras use can use to see your environment. It technically has VR, AR, and MR capabilities but is still labeled as VR because that’s its primary function.

Recent History

This technology has been developing for decades with the first headset made in the late 60s. Nintendo’s Virtual Boy and Google Glass are infamous examples of pushing the technology before it was ready. It wasn’t until the last ten years that we saw this tech being offered at standards deemed acceptable by early adopters.

Oculus/Meta, Valve, HTC, and PlayStation are the biggest producers of XR headsets. All except Valve have released multiple headsets since the mid-20-teens. Initially they required external tracking sensors that would have to be set up around users’ rooms and needed to be plugged into a machine to operate. Newer headsets no longer need external trackers, have built-in computers, and can operate on battery power.

Microsoft’s HoloLens was announced in 2015. It continues to cater more towards businesses and focuses on specialized use cases for job sites. The HoloLens team was reported to have downsized during the series of layoffs in the first half of 2023.

Current State of the Market

In the VR space, there has been an emphasis on mobility and breaking down barriers to entry. Eliminating the need for beefy PCs and lowering the cost of headsets by almost half has made VR more accessible than ever.

Meta and Apple are the two biggest contenders for AR/MR. Meta’s Quest Pro has been on the market since last October and Apple is getting ready to release their Vision Pro early next year. Both offer wireless MR with only 2 hours of battery life. Besides access to its ecosystem, Apple's biggest advantage would be the superior hand tracking it is advertising. However, Apple’s headset is about 3.5x more expensive than Meta’s headset at $3500.

Current Use Cases

Gaming is by far the most common use case for these headsets. Look at the marketing for the headsets currently for sale and the overwhelming majority focus on the gaming element. I own an Oculus Rift S and when I open the Oculus app, I see hundreds of games and maybe a few educational apps.

Communication is the second biggest use case advertised. Videos of people having meetings in the Metaverse or people talking in VR Chat are often showcased when talking about this tech. The problem with these use cases is that they are needlessly complicated. The technology is there but there are not many practical reasons to use XR over a traditional platform like Zoom or Teams except for the novelty of it. A case can be made for some jobs where it would be useful to experience the physical scale of an environment or object.

What will make it mainstream?

XR headsets are more popular than ever, but I expect their demand to increase dramatically over the next ten years as the technology continues to develop. The addressable market for XR headsets is potentially anyone who can afford a smartphone. But everyone who has a smartphone doesn’t want an XR headset right now because there is no universal benefit to owning one. A real impact on everyday life must be observable for the general consumer to be interested.

I foresee a product similar to the HoloLens being the first headset able to gain mass adoption assuming it has an observable benefit, a reasonable price point, and a battery that could last most of the day. Unlike most headsets, the HoloLens design doesn’t cut off peripheral vision, and the real world isn’t displayed on a screen. Both of those factors seem optimal for extended use. While a product like NReal’s AR glasses looks like a perfect model, I don’t see something with such a small form factor coming to market with enough features to warrant mass adoption. This baby needs power.

Features of a mainstream XR headset

Going back to the definition of XR, these screens need to have an impact on your reality. It isn’t enough to have a screen on your face if it’s only as useful as the one you have in your pocket. Focusing on the unique nature of XR headsets, we can narrow our focus on what benefits would be most appealing to consumers.

1. Features based on the headset’s ability to see what you see.

I wake up and put on my headset. I walk to my closet and see a pair of jeans and a T-shirt that have an overlay on them to get my attention. My headset knows what clothes I usually wear. It picks this outfit because I’ve worn it before but not recently. I look over in the mirror and see myself not in my pajamas but in the suggested outfit. If I decide I don’t want to wear it, then I can ask for another choice or give it a suggestion. “I’m walking a lot today, maybe something with shorts?” I briefly take off my headset to get dressed and when I put it back on it lets me know that it has some suggestions about the organization of my closet. I have everything sorted by color and it isn’t super dirty so I’m curious what it thinks I can do better. It’s not talking about my clothes but the stuff I have on my shelves. Some old books, shoeboxes stuffed with old electronics, and sentimental junk I’ve collected over the years. My closet is transformed before my eyes and I’m looking at a layout that makes it easier to access what I use most and save space in the process. It can do this with almost everything including my desk, backpack, and kitchen. I quickly reorganize my closet with its assistance before heading walking to the door.

2. Features based on the headset’s ability to see what you can’t see.

Before I walk out, I look at my pantry. I can see what’s in it as if I had X-Ray vision. I’m seeing what the headset remembers from the last time I opened the pantry door. I couldn’t remember if I had any Zebra Cakes left. My headset says there weren’t any last time I checked. I’ll have to go to the store later. I start to walk to my car, but the headset tells me that I forgot to grab my keys, wallet, and phone. My keys and wallet were on top of my dresser like usual, but I don't see my phone anywhere. I see a semitransparent arrow that guides me to my phone. I get in my car and start my drive to campus. I think I see my friend driving behind me. I get an alert telling me to look in front of me because the headset sees I've been looking at my rearview mirror for the past ten seconds. I’m thankful for this alert because I almost missed a stop sign. There’s construction so I have to find another route to my parking lot. The headset knows I’m on my way to class and it starts to give me directions. I can see arrows projected on the streets telling me where to go. I see a list of the next couple of directions in the upper left corner of my lenses.

3. Having the headset remove what you don’t want to see.

After parking, I walk to class. I pass by people wearing a similar headset, but I can’t see it on them. That’s because I’ve seen their face without one so my headset knows what their faces should look like. When I get to class, I look at the whiteboard. Last semester someone used a bad market on it, and it left a permanent smeared stain. My headset knows it’s a stain though and removes it, so I don’t have to look at it. It makes my world more colorful, removes small blemishes from my environment, and even hides unwanted acne.

4. Adding to what you see.

Looking at my peers I can see little messages by their faces. One guy's message reads, “Studying for Finance exam today.” This could either be a potential conversation starter or a warning to not distract him. I want to listen to something before class starts. I put in an earbud and a media control panel appears on my desk. I open my music library and pick a song.

After class, I go to the gym. My headset knows my routine and gives me a list of exercises to complete for the day. I don’t know what one of the exercises is, so my headset shows me how it’s done. An avatar, taking the role of my personal trainer, appears in front of me and shows me how to do the exercise. My headset monitors my movements and corrects me until I have it down. As I’m cleaning the last machine I used, someone comes up and asks if I’m done. “It’s all yours.” They were speaking a different language, but the headset gave them subtitles for me to read.

Conclusions

While what I described might sound like the beginning of a dystopian science fiction movie, it’s where I think XR headsets are heading. I didn't even mention the privacy concerns and other societal hurdles that need to be jumped until our lives look like this. I hope we're closer to this reality than most people think we are.