Or listen on your favorite podcast app
Apple Podcasts / Google Podcasts / Spotify
—
Augmented and virtual reality will change how we see the world. Here’s how three digital futurists see the near- and long-term potential of AR/VR.
Hear from Wired magazine co-founder Kevin Kelly, Contextere founder Gabe Batstone, and Oblong Industries CEO John Underkoffler.
Continue below for a written version of this podcast episode. Quotes have been edited and condensed for clarity.
—
Video games have been a staple of entertainment for decades. Across all devices and through ceaseless evolution, they almost always have a few things in common. Players are shown the playing field, like a puzzle or a virtual world to explore, and information at the periphery. Maybe it shows how many lives you have left, or a map or inventory. That information is there to make the game more playable, winnable, and fun.
Now stretch your imagination; what if your life had the same thing? What if you could pull up any info you need at any time in your periphery to complete any task? What if it made your job more like a game? What if it made life more playable, winnable… and fun?
It may be closer than you think, according to author and futurist Kevin Kelly. He’s spent a lifetime studying the broad-strokes implications of technology. Among other things, he’s an authority on immersive technology like Augmented Reality, or AR, which overlays information on a user’s field of view, and Virtual Reality (VR) which completely fills the users’ view with a digital space and virtual world. Recently, he noticed an emerging trend worth exploring.
“Just as the web was this immense platform that enabled lots of things to happen, and then we had smartphones and social media, which was another big platform, this is going to emerge to be the third stage, the next platform after smartphones, smart glasses,” says Kelly.
“Smart glasses” have become a common notion in the zeitgeist, showcased in movies like Ironman and Spiderman. And they’re more technologically feasible than ever before. But Kelly felt a less literal name was needed to identify this type of tech and other tech like it. Several names were in the running; you might call it the “AR Cloud”, or “the Metaverse”, but the name Kelly uses… is the Mirror World. And, like the internet or smartphones, he’s predicting that the Mirror World will impact every part of our lives.
“We’re at this point where even the inventors of this technology don’t know what it’s really going to be used for. But there are a number of different, what I would think are immediate uses.”
In 1989, Kelly was one of the earliest to experience what we now call virtual reality, at the lab of Jaron Lanier. It was Lanier who coined the term Virtual Reality, and his company sold the first VR headsets.
“I saw VR, virtual reality in the 80s and I thought it was going to be [here in the] next five years,” said Kelly. “And so I was like, ‘Oh, this is around the corner.’ Well, the problem was just the price. Those setups back in the eighties cost a million dollars.”
Despite brilliant technology, the gear was too expensive and bulky. It would take more than twenty years before immersive technology would again surface in a big way. Smartphones made that possible thanks to their efficient processors, high-resolution screens, and tiny sensors. Throughout the years, Kelly has continued to sample immersive technology, trying every major system there is.
The way Kelly sees it, the Mirror World is a new kind of Internet that we’re already starting to see. It’s virtual worlds overlayed on top of the real world, woven together to create a massive one-to-one map of the globe with data embedded everywhere. The Mirror World is viewable through augmented reality, virtual reality, and everyday surfaces that double as screens. Imagine seeing all the information from Google Maps, like navigation, reviews, and nearby restaurants, but rather than viewing it on your four-inch smartphone screen, the data is visible all around you. It’s useful data deployed everywhere you look in real life.
Maybe in the not-distant future, you have a smart mirror, a very literal take on Kelly’s Mirror World. You find a new garment online, and in the mirror, you see it on your body. You adjust the cut, the style, the color, get it just right. You turn around and see how it moves with you, noticing your skirt flowing in the virtual wind. Or maybe fashion isn’t your thing. You’re more of an architecture buff, visiting Chicago on a business trip. Anywhere you go, you could summon the history and details about the diverse structures across the skyline, without once looking down at a smartphone. This technology could help with a home remodel, rearranging furniture, moving walls, trying different shades of paint. It could help in the kitchen with new recipes. The uses aren’t just domestic; think about this in the context of surgery assistance, real estate walkthroughs, or classroom instruction. The possibilities are endless, just like the Internet. But the area that could see the most immediate impact is one usually seen as low-tech and low-fun: industrial blue-collar work.
“I think the first users are going to be in occupations where they need to do repair or training… And the way this works is that… you see the real machine and you have this virtual overlay of what is what and where things may be broken and what wire to move over here… So they see this overlay of the real thing and the virtual understanding of it, and this is actually not theory. This absolutely works and is being used today.”
When Kevin Kelly says this technology is being used today, he’s talking about companies like Contextere.
“There is a long time ahead where people need to put warm hands on cold steel, right? They need to install, maintain, repair and inspect complex equipment. And that’s not going anywhere,” says Gabe Batstone, founder of Contextere.
Contextere is a startup building machine learning algorithms that extract previously inaccessible data for industrial environments. Batstone is at the forefront of innovation trying to deliver the right training tutorials and information at the right time and place.
The way Batstone sees it, it won’t be long until every job is a data job; whether that data is served up through a smartphone, or “smart glasses”, or other wearable devices and augmented reality systems.
Batstone puts it this way:
“Computers are very good at analysis and they can provide you with a series of potential decisions in a given moment. But the reality is, humans understand judgment. And that’s putting a decision in context.
“In the industrial environment where the pace of technology moves so fast, the equipment is being upgraded constantly and changing. It is literally impossible to keep all of the training, and even operational materials, up to date at every moment. If you lessen the burden of trying to train everyone for every possible scenario in advance, which has become impossible, and just say, ‘No, on the job, let’s provide access to intelligence.’
“So we’re kind of whispering in people’s ear as they go about their job, like ‘be careful that’s hot’, or ‘just a reminder that six months ago you worked on a similar procedure and perhaps you’ve got it wrong, so you might want to slow down.’ And I think this is one of the most powerful abilities that we’ve created, is the ability to actually talk to the equipment.”
This tech is still young, but the potential impact is huge. It might be easy to fixate on the hardware aspect, debating whether the completely immersive virtual worlds of VR will be the winner, or maybe the heads-up-display overlay of AR will come out on top. But Batstone is quick to point out that it’s not really about one device or another. It’s about enabling ready access to information.
If any of this sounds too far-fetched, too sci-fi, too “Star Trek Holodeck”, just remember; there’s already a version you’ve probably used. When was the last time you needed to learn something… and you found a YouTube tutorial to guide you?
There’s a fundamental reason why this type of medium is so effective as a teaching tool, and it’s because when it’s working, this tech plays a funny trick on your brain. With virtual reality, you might consciously know you’re not on an alien planet, or staring over the edge of a skyscraper. With augmented reality, you know that names, numbers, charts, and tables aren’t really floating in the air around you, guiding you as though it were a very realistic video game. But your limbic system, what many people call your lizard brain, that part of your brain… can’t tell the difference. Your stomach will drop looking over the edge of that virtual skyscraper. You’ll feel it, even if you know it’s not real.
And the same things that make video games so compelling could be built into any aspect of your life — shopping, tourism, exercise, self-improvement, your job. It could make meaningful, productive work… like a game — genuinely interesting, fast, and even fun.
In the 2002 film Minority Report, Tom Cruise plays Chief John Anderton of the Washington DC PreCrime police. The controversial law enforcement unit uses psychics to see crime before it happens, and the murder rate in DC falls to zero. The movie was a box office success and was predictive in incredible ways.
Looking at Minority Report today, you’re struck by the incredible foresight it had in predicting new technologies. Its representation of targeted advertising and self-driving cars were years ahead of its time. It showcased virtual reality in a believable way long before VR had made a resurgence.
And the scene that stands out most… is the gesture-controlled computer that Anderton uses at the beginning of the film. When the psychic system signals a possible murder in the near future, Anderton summons data on his massive display, sorting, scaling, and sequencing images as though he were a conductor before an orchestra.
“Quickly after the film came out, I started getting phone calls from Fortune 50 companies basically saying, ‘Hey, that stuff in Minority Report, is it real?’ It was an opportunity to get these ideas out into commercial form where they can actually do some good, where they can be used by the world of people who design things and make things and make decisions and shape the world,” says John Underkoffler, the man who invented the iconic computer interface from the film.
Underkoffler was part of a think tank of 15 eminent technologists gathered by the director Stephen Spielberg. The group included Kevin Kelly and virtual reality pioneer Jaron Lanier. They helped steer the futuristic elements of the film, and are largely the reason it was so prescient.
Even before Minority Report, Underkoffler was convinced that the Graphical User Interface, or GUI, we’ve used in computers for decades was long outdated. After Minority Report, Underkoffler founded Oblong Industries, to turn his fictional gesture interface into reality. He continues as CEO at Oblong today
“I’d spent a year of my life trying to build a coherent vision of the technological future that would exist in this speculative 2054 Washington DC… and in particular, the view that I spent so many hours trying to dial in on [was] how a user interface could make a small team incredibly effective. Out of the many problems that the design effort behind the film had to solve was what is the future of computation. And Spielberg himself said, ‘Please don’t tell me we’re still going to have a keyboard and mouse in 50 years.’”
Like Kevin Kelly and Jaron Lanier, Underkoffler has been working on this technology for decades — far longer than casual moviegoers might guess. And like Kelly and Lanier, Underkoffler is eager and ready for the paradigm change.
“The GUI that we have on our laptops is still good enough as a general-purpose GUI that we can do what we need to do. We can do it as programmers. We can do it as finance people. We can kind of make a go at it as designers and so forth. The tools aren’t horrible, but the UI itself is still fundamentally holding us back. And it’s not a matter of quantity. It’s a matter of quality. There are things and thoughts and ideas and kinds of work and kinds of synthesis that we literally cannot get at because of the UI.”
Imagine being able to virtually confront, control, and use all the data around you – and on the internet – to help you accomplish your goals. Once you can visualize the next step and utilize real-time information via a heads up display, suddenly, jobs become easier. Tasks can be automated. Safety protocols – are now verified in real-time.
The world is full of people looking for help in their work and personal lives. They’re in every corner of the world, speaking many different languages, but ultimately they’re looking for much the same thing; the chance to do good work, provide for themselves and their families, connect with other people, and enjoy themselves along the way. The Internet has helped billions do those things. Smartphones too. And Mirror World technology is poised to take it further.
What if the secret to making our lives and jobs more playable, winnable, and fun… was hidden in plain sight?
