While I’ll beg your forgiveness for the obvious hype-train connection in my title to the release of the film READY PLAYER ONE, the film does raise many questions about the future of Virtual Reality (VR) and Augmented Reality (AR). For me, this question often comes up in the context of VR and AR in education, since my work as an Instructional Designer often means looking at new technologies that could change the way we go about the task of learning and teaching. Our office recently started a working group together with partners across the university to begin exploring what VR and AR in education could look like.

Now, I’ll have to admit that I haven’t read the original by Ernest Cline, nor have I seen the film (yet), but I do have some familiarity with how these types of stories play out: from Tom Clancy’s Net Force series to other films like The Matrix series, these fictional takes on virtual worlds often seem to be more dystopian than a revolutionary technology should be.

READY PLAYER ONE seems to be no different, with the back story being that the real world is in ruins, gripped by an energy shortage, and many people regularly seek the refuge or retreat of the virtual world known as “The Oasis” for entertainment, community, and commerce. The action then revolves around a group of teens trying to save both the virtual and “real” world from an evil corporate takeover while searching for an “Easter egg” hidden in the virtual world that gives great wealth along with total control of “The Oasis” (those more familiar with the book and/or film will have to forgive my clumsy retelling, since much of my understanding is gathered from reviews or the trailer).

So, why all the backstory on a bunch of dystopian takes on VR and AR, and what are we doing to approach the topic given all this negativity? Well, we’ve taken a similarly cautious approach to our explorations, since we can see just how wrong these approaches can be through the warnings offered to us in dystopian literature and films.

Keeping these bleak world-destroying possibilities in mind, these are some of the considerations we are grappling with:

First of all—why bother with it?

Having spent some time with a few VR systems (Microsoft Mixed Reality headset and Google Cardboard/Samsung Gear), I can clearly see why these devices and experiences could lead to huge shifts in educational settings. These devices allow you to feel like you are in the middle of things—whether that be a physical location or “field trip” scenario, or inside a simulated beating heart to view the effects of high blood pressure or heart attacks—they can bring a new connection, observation, and feeling to the materials. It makes it feel real—at least more real than words or images on a flat page. They move in ways that models or replicas can’t and can be shared across time or spaces (i.e. students don’t need to be in class that day, or can go back and review things, whereas a physical model may be put away or moved to another class).

Speaking of moving through time and space, VR also allows us to recreate historical moments or locations that don’t exist anymore—like witnessing the Battle at Gettysburg or the signing of the Magna Carta. We could stand in the markets of ancient Rome or sit in the audience at the Globe Theatre watching Shakespeare’s productions. Some of these things we can visit during recreation days with local theater companies or volunteers, but often don’t have the same scale and impact as experiencing a large scale VR view that we can move around in.

Recreating elements and experiences in virtual worlds can also greatly reduce the cost and time of prototyping (and adding a new kind of rapid prototyping). Think about a set design, hotel management, or mechanical engineering course—students would be able to build sets, lobbies, or prototypes in a virtual world, tweaking them as they move through them or simulate their function and flow, seeing what works and what doesn’t. An architect (or architecture student) could give the client a virtual walkthrough of the space they’ve designed, simulating things like light and shadows, adding in the sights and sounds of water features, or showing the reflections or views of nearby buildings, all in a virtual world—this could hardly be recreated in drawings or tabletop models.

VR and AR can also help to raise awareness of humanitarian situations without putting viewers at risk. We’ve seen The New York Times and Life, among many other publishers, begin moving into 360° videos of war zones, natural disasters, poverty-stricken neighborhoods, and prisons. These are places we wouldn’t necessarily expect students to visit regularly, but could show them the realities of through virtual experiences (while recognizing that, long term, having any real impact in changing these things will at some point require people to make actual trips to these places). Seeing these experiences brings the reality closer to their studies—that what they are reading about and discussing in class often isn’t a theory or thought experiment, but for many people constitutes their daily living. This helps to build empathy and purpose into their learning, leaving them better prepared to handle these situations when and if they do visit these places to work toward change.

AR experiences are a bit farther off but have the same potential to help make concepts relevant and accessible to students. Augmented Reality would allow informational overlays on the real world, giving context beyond what can be observed directly. This approach would allow environmental studies students to view plant and animal names while visiting a forest ecosystem, or for a computer networking student to see information about a server’s temperature, the wiring path of an ethernet cable, or WIFI signal strength in a crowded environment. Students studying abroad could have real-time visual translations of the street signs, local currency conversion rates, or information about what route to take or what areas to avoid. To my knowledge, no AR system exists that is this advanced yet—though the Microsoft HoloLens and other AR systems do have the potential for such future applications, given the right developers and enough processing power to run these.

The shortest way to put it is that VR and AR open experiential learning and observation in ways that are inaccessible to some students (given the prohibitive cost of traveling abroad to visit some of these places) or impossible (after all, Ms. Frizzell and the Magic School Bus can’t really take us inside a beating heart). It takes things that are difficult or costly to present physically (like buildings, sets, or engine parts) or observe (like the path an ethernet cable or WIFI signal takes through a wall) and makes them easy to see before spending on the production.

That sounds great—so what about all this dystopian stuff?

While our working group hasn’t gone so far as to see these technologies as potentially world-ruining, as they are in READY PLAYER ONE, we are well aware of what we would see as dystopian (or at least problematic) ends in an educational setting. A few of those potential negatives are even on the other side of one of the positives—experience and cost—among others.

Experience

One worry is not presenting these technologies as replacements for the real thing, at least not in all cases. Visiting a virtual war zone or natural disaster to demonstrate the humanitarian crisis for civilians may help to facilitate empathy, but as noted above, eventually someone needs to be willing to be on the ground to help rebuild. Viewing a virtual open-heart surgery may help nursing or pre-med students understand the procedure and steps involved but it won’t help them gain the dexterity of using a scalpel.

Likewise, visiting ancient or historical sights might be interesting in VR, but will often pale in comparison to actually standing in the ruins of the colosseum or seeing a local theater production of Shakespeare. Viewing a virtual world might help us prepare for travel abroad, but what will really matter in a student’s experience is the actual study abroad trip and the interactions with local people, culture, and food.

We see the VR and AR experiences as early steps on a path or as part of a linear progression building up to the more dangerous or costly experiences. An aspiring nursing or pre-med student could re-watch or reperform the virtual heart surgery 100 times as they build familiarity and awareness of the procedure, whereas practicing on a cadaver or viewing a real procedure may only be possible once or twice in their studies, or for only a few students at a time. Similarly, a student might feel more comfortable navigating the streets of Paris or recognizing significant locations after having seen it in a virtual setting a few times before.

The point here is that we are introducing or reviewing concepts to students through VR and AR, not using them as complete replacements for the real thing. Having limited exposure to an experience might not lead to the deeper learning we hope for, so having additional time in VR and AR can help create deeper memory and recall of these events both before and after the real thing. Using something like a 360° camera, a student might even be able to capture their own travels or interactions, acting as a kind of virtual reality notebook of their actual experiences and visits that they can view in virtual reality later on—something which right now is not possible with traditional note-taking.

Cost

While experiencing virtual worlds or prototypes in VR might be much less expensive than traveling abroad or producing a prototype part or set, there is still a prohibitive cost element. Not everyone will be able to afford the equipment needed for some of the high-quality experiences or the hardware or bandwidth requirements needed to access them. Most of the VR hardware I’ve used costs thousands of dollars for the full setup of a headset, computer and graphics card, and high-speed internet connection. Even Google Cardboard requires a smartphone and fast WIFI or LTE connection that adds up to a few hundred dollars at least.

These prohibitive costs need to be managed—either through somehow subsidizing/renting out devices students would need to purchase for a class, or by simply having the university cover the costs while maintaining and managing the devices in a lab or classroom setting. It also means that a future where these technologies are both an on-campus and off-campus experience is much farther off than we might like (since most online students wouldn’t appreciate being required to buy a $3,000 HoloLens for their course, they’d have to come to campus to use such technologies). While most students have smartphones and could access Google Cardboard or similar tools fairly inexpensively given that they already have the phone, we then run into our next and biggest challenge so far: content.

Content

This area brings together the two previous challenges in a few very important ways. Finding relevant content that can give students meaningful experience has so far been a challenge. While we might find an app that simulates the heart and various conditions, that may only constitute a small part of a student’s coursework. The ones that do lend themselves to educational settings feel more like demonstration or proof-of-concept programs rather than anything full featured or ready to debut in a course. So far, VR and AR are still in the novelty stage (also known as the “early adopter” stage) of the technology development curve, meaning many of the experiences available on these devices are games or one-off experiences that wouldn’t fit in any university classroom yet.

This also means that providing this relevant content can be very expensive, and thus cost prohibitive for students and even universities to purchase, since it would likely need to be custom designed for the courses using them to ensure both breadth and depth of content. Creating this content would require teams of developers and content producers to get it done in any realistic timeframe.

Finding experiences and elements that are both meaningful and accessible (both in terms of cost and hardware/system support) is the biggest challenge we’ve encountered and is one we are still working through as we discover new hardware, developers, applications, and providers. Over time as these technologies and approaches become more widely accepted, costs will fall and content availability will rise, but we aren’t yet at a time where that outcome has a clear date. Some of the major publishers have announced plans to provide VR content, but most seem to be focused on K-12 markets at this time.

System Compatibility

Another major challenge is simply getting an AR or VR setup that would work for most students.  It wouldn’t be realistic to tell students who use MacBooks or Chromebooks to go out and buy a Windows 10 system to access Oculus, HTC Vive, or Windows Mixed Reality content, or to tell an Android user to go out and buy an iPhone to get ARKit (or vice-versa). 

System compatibility at these early stages is often a format war—the two competing standards or formats grow side by side as competing factions with no hope of interoperability between them.  Eventually, one winner takes all the market share and becomes the de facto format while devices running the losing format are relegated to the tech junk heap, or become little more than curios for the early adopters’ tech collections.  Think back to the Betamax/VHS battle in the 70s, the MP3 vs OGG or MPEG4 war in the 90/00s, or DVD vs Blu-ray in the 10s.

VR and AR in the early stages is a bit of a walled-garden without mixing between systems—two of the leading standards are based on Oculus using the OVR API while HTC Vive runs on Steam VR. There is increasingly more crossover in this area, and it seems that is largely because of the ubiquity of Steam as a publishing and distribution platform. This early uncertainly means that, for some, the VR format war also runs the risk of becoming a winner-take-all in the end, which is good in the long term for mass adoption but can be incredibly problematic and expensive for those who have already invested time and money into the losing system. In most cases, it is best to wait and see which format will win out, or second best to hope that the developers offer interoperability or backwards compatibility once a forerunner emerges.

Other considerations

Beyond the specific items above, there are other concerns presented more broadly—and these are likely where the dystopian books and films get their inspiration. Things like user privacy become an issue as users begin to integrate the virtual worlds into their real worlds. While having an augmented reality overlay might give an individual the ability to bring up information about the scenes and spaces in front of them, it would likely also provide advertising companies the user’s location and profile information, as well as information about some of the people around them, leading to targeted advertising or other user’s profile information appearing over the user’s information display. As horrifying as pop up ads were on our computer screens, imagine them in the real world, invading every view you have.

Keiichi Matsuda’s short film HYPER-REALITY provides a poignant look at an augmented future gone wrong.

We also live in a world where bandwidth caps are a reality (at least with many U.S. based ISPs), meaning we have limitations on how much we can download in a given period. VR and AR experiences are likely to require a lot of streamed data, since the files associated with them are either updated constantly or too large to store locally. This would present a challenge for a student taking an online course with VR and AR aspects since viewing the experiences a few times, coupled with all the other traffic on their network from family, friends, or roommates could lead to their connection hitting a bandwidth cap, resulting in higher costs or slower data, the latter of which would render most VR/AR experiences unusable, and either of which could put the student’s ability to complete the course at risk.

So, what’s next then?

At this point we are still in the cautiously-optimistic exploration stage while we wait for the early adopters to help drive development, and tech companies to reduce costs and increase accessibility (mostly in terms of support for more systems/OSes). Our working group is currently gathering information on which developers and applications are producing content that is relevant to our colleges and departments, as well as tracking developments in hardware. We do have a few devices to sample some of the early content, but are also reaching out to campus partners who have access to other hardware or are developing applications themselves (such as our College of Computing and Digital Media).

Overall, the biggest consideration or challenge is avoiding the pitfalls outlined above—after all, the real point behind dystopian literature and film is to take caution about the future we allow or pursue. Having the foresight to predict these issues is the first step in avoiding them, which is the entire reason our working group started from an information gathering stage rather than getting pulled into the hype and inflated expectations.