The Next Frontier for AR? Your Ear
When augmented reality goes mainstream, you’re most likely to experience it through a product you already own — headphones.
When people think of augmented reality, they think of it this way: What if your life were a screen? What if your entire field of vision were a big heads-up display, interposing the virtual upon the real, and blurring the line between both? And because this is what people expect from augmented reality — a Minority Report meets Pokemon Go–type experience they can never shut off — they are rightly skeptical. Because who at this point really wants another display in their life, let alone to turn their whole life into one?
Yet chances are when augmented reality first goes mainstream, it’s not going to be because of a Google Glass–like product that places a heads-up display in front of your eyes. That may still come (and there’s some degree of evidence that companies like Apple are already working toward such next-gen AR hardware solutions), but before it does, you’re most likely to experience AR through a product you already own: your headphones.
Who at this point really wants another display in their life, let alone to turn their whole life into one?
One company laying the groundwork for an audio revolution in AR is Bose. This year, the Massachusetts-based audio maker has quietly gone whole hog on AR. They have not only developed their own toolkit called Bose AR to create audio-centric augmented-reality apps, they are also committing to make every pair of headphones they make going forward compatible with AR. Which effectively means all their headphones will be Bluetooth and contain embedded sensors: a magnetometer, an accelerometer, and a gyroscope — the same sensors that determine everything in your smartphone, from the direction in which you’re heading to whether you’ve placed your handset face down.
Bose is currently shipping two AR-compatible products: the QuietComfort 35 wireless headphones and the Bose Frames. They’re both great products, but the Bose Frames (which Huge, Magenta’s parent company, collaborated on) are perhaps more interesting, simply because they eschew the usual form factor of headphones. Instead, they are a pair of AR-equipped sunglasses, with special noise-canceling speaker technology built into the earpieces, so they double as headphones.
In action, the Bose Frames give a good glimpse into what the audio-driven world of AR could look like. By pairing the sunglasses to your smartphone, you can use your Frames to run apps that give you information only you can hear about the world around you. For example, you can look at a restaurant, double tap one of the ear pieces, and be told its average Yelp rating; look at a statue of John Endecott in Boston, and you might get a history lesson about how the early Massachusetts governor liked to lop off the tongues and ears of his enemies. There’s even games you can run on the Frames, like Sonic Samurai, which drops you into the aural perspective of a blind ronin, dodging katanas and shurikens by sound alone as they whiz by your head.
And those are just some of the possibilities of audio AR. Imagine headphones that can pick one person’s voice out of a crowd, or translate what a Japanese speaker is saying into spoken English in real time, or a Shazam app that can tell you what a song is with just a nod of your head. All of these examples aren’t futuristic. They’re doable now.
According to Bose’s principal augmented-reality advocate, Michael Ludden, AR makes more sense on headphones than screens because the technology is already widely available, and because audio is just less intrusive from a UX perspective.
“I feel implicitly that we are reaching critical mass on how many screens we can have in our lives,” he says. “Right now, as I’m talking to you, I have four screens in front of me: a laptop, a smartphone, a tablet, and a smartwatch. That’s too much. We would do well to remember that we have five senses that can be augmented, and that no one thinks a good vision of the future is for everything to have an insane amount of visual information overlaid upon it.”
At Huge, audio AR is also seen as a big growth area over the next few years. Chief Design Officer Derek Fridman sees sensor-equipped “smart” headphones (like Bose, Apple’s AirPods, or Google’s Pixel Buds) as a bit of a Trojan horse for AR, just waiting to have their true power activated. After all, these headphones already spend all their time connected to an AI-equipped computer in your pocket and have a direct pipeline to your brain, even when you’re not otherwise on your device: Why shouldn’t they allow users to glean contextual information about the world, too?
There’s exciting business possibilities in audio AR, too. Fridman tells me about one experiment Huge did internally at its coffee shop, Huge Cafe, on Peachtree and 17th in downtown Atlanta. By pairing computer vision and machine learning with AirPods, Huge’s baristas were able to not only be clued in to every customer’s name as they approached the counter, but even make an educated guess as to their regular order. It’s the kind of thing that sounds like science fiction today but will be as common as your local coffee shop in the future, Fridman argues. “Everyone is racing to replace their employees with robots,” he says, “but we’re a long way away from it happening. So why not enhance the employee instead?”
None of this is to say that a visual AR headset like Google Glass won’t eventually become mainstream. Fridman points out that if you piece together the components from many of the new products Apple is selling — the Apple Watch’s tiny, cellular-equipped system on a chip, the AirPod’s sensors and audio technologies, and the Apple Pencil’s long cylindrical batteries perfect for embedding in earpieces — they could easily become the ultimate AR system. But such a product is likely a few years away, from both a technological perspective (in-eye displays are still imperfect) and a cultural one (think of all the dorks wearing Google Glass you’ve nervously avoided or made fun of).
Even after such a consumer headset emerges, though, it might just be that audio is the preferred way people have of interacting with AR, just like voice assistants have become the default way people interact with their automated smart homes. And from that perspective, both Ludden and Fridman agree that headphones are already perfectly positioned to deliver genuinely useful AR experiences, in a way that the Pokemon Gos of this world still aren’t.
“There’s zero chance that augmented reality isn’t going to take off in the next few years,” Fridman says. “And when it does, the first mainstream killer app might well be audio.”
Magenta is a publication of Huge.
