Learning spatial awareness: virtual reality and 360-degree video

Howard Errey is a psychologist with an interest in innovation. He works as a Senior Coordinator, Digital Learning, for the College of Design and Social Context at RMIT University and blogs here. Here Howard shares a fascinating journey into the world of virtual reality and 360 video to offer an immersive learning experience.

The Problem

At RMIT we have a common problem in a number of subject areas. Educators in areas such as Property Construction and Project Management, and Architecture and Design, need to provide building site experience for large numbers of students – in some cases hundreds per class. The practicalities of turning up to a major building site with large numbers of students for reasons such as safety, make these site visits increasingly difficult to organise.

Credit: droidcom Berlin 2015 by droidcom Global

There have been National VET projects in the past that made admirable progress with such problems. Stefan Schutt and his team at Victoria University set up some great 3D design projects at The Lab with former National VET E-learning Strategy and Australian Flexible Learning Framework funding. Back in those days SecondLife was all the rage and it gave an immersive learning experience with a sense of fun. It is great to see that The Lab (and its wordpress site), which provides computer training including 3d modelling for young people with Autism Spectrum disorders, is still going so strong.

What RMIT have been doing

With staff from our E-learning Innovation Incubator, we approached the problem with 2 hypotheses:

  1. that we could easily create 3D virtual reality objects for an immersive experience, and
  2. that using 360-degree photography and video would make a useful addition to that experience.

Firstly, with the availability of Oculus Rift (OR) developer kits, it seemed obvious that 3D objects could be built that recreate realistic experiences. In 2014 we employed students from the RMIT Centre for Games Design Research to show staff how to build objects in 3D software engines such as Unreal. This generated a lot of enthusiasm. The challenge we discovered was enabling staff enough time and computing power to make this easy.

In 2015 we tried a different approach. At that time students in these subject areas learned to build 3D designs in Google Sketchup and Revit. We decided that it would be pretty cool if we could make it easy to take a design from one of these programs and turn it into a 360 immersive experience in the OR. We again employed a student from the games lab, who this time developed a series of workflow documents and videos. Using IrisVR, 3D designs can be dragged and dropped into the program which turns them into a 360-degree version. It’s not quite as simple as it sounds (this software is still in early development) but it is now something students can do with their work. The next challenge is providing powerful enough hardware, perhaps in the library, where students can have the computing power.

What about Cardboard?

Given the challenges of design for the OR and the low cost, why not just use Google Cardboard? As a baseline this makes a lot of sense as development costs of learning objects (similar to Google Expeditions) are also low and potentially fast (more on this below). OR has a number of advantages, however, will be widely available at a low price point soon, and shows potential such as distance collaboration in shared virtual spaces, so it needs to be kept on the radar. In particular, whereas in Cardboard you view from fixed points, OR has the potential for the experience of moving through 3D simulated environments.

There is a great thing about Google Cardboard. By getting teaching staff and students to physically put together the goggles, with their hands, from a pre-cut kit you can purchase for under $10, a number of remarkable things occur. Firstly, there is a sense of self mastery in making something that is reasonably easy. It continues to pleasantly surprise me the shared delight that arises, when a group of staff or students get hold of cardboard, put it together and then share what they experience. Secondly it provides a small but powerful innovation experience. Something changes in the brain. Compiling something new from different small parts (a cardboard ‘box’, lenses and your phone) is a small scale model of rapid prototyping). On a larger scale, Cardboard models how Google Apps in Education works – not as a complete (or expensive) system such as an LMS, but rather showing that you can put together different small parts (in particular the Add Ons in the Google Apps ecosystem) to make something new. It is both a small example as well as a metaphor that anchors Google ‘thinking’ as well as innovation in the mind. They’re not stupid over at Google.

Credit: cardboardx by howard61

360 Degrees

This is where recent developments have also made creating objects easy. With our project we thought that creating 360-degree photos and videos of building sites would be a simple and a low-cost entry to creating immersive experiences.

For 360-degree still images, the Google 360 app on your phone lets you point your phone around an environment and it automatically stitches together the spherical image. A nice example is here. For video we initially looked at GoPro cameras. There were a few solutions we saw online, in particular 3D printing a mount that holds six cameras filming simultaneously. The challenge is to then stitch together the six videos, which seemed complex and expensive, and the GoPro stitching solution was not yet available.

In the meantime, the latest Ricoh Theta S camera was released during the project. This camera provides an experience similar to the Flipcam video camera, in that it simplifies a single function seamlessly into one device. The Flipcam was so good that Cisco bought it and closed it down because the live streaming potential presented such a threat to their core business model, but that’s another story. With two opposing bulb shaped lenses, the Theta S creates 360-degree video at the press of a button. Plug it into the computer and upload it into YouTube, which converts it to a 360 format ready for viewing in Cardboard. The whole process takes about 15 minutes. Our first example, a walk around the office, is here, which we recommend viewing in Chrome. You can click and drag the mouse on the screen to move around the spherical view as the video plays. Although the Theta S camera is high resolution, we learned that the resolution is spread around the 360 degrees, which does not make it super clear. It is still an awesome, useable starting point for learning how to work with 360 video and, at under $500, well worth the experiment for what you will learn.

What we learned

  • It is worth persisting with the OR. While technically challenging it provides the current conceptual lead for developments in 3D immersion.
  • Think students first. If we can manage to build something, they can, provided we design and scaffold the learning.
  • Create a minimum accessible standard of experience and technology for students. I can hear Bronwyn over at E-Standards for Training nodding in agreement!
  • 360-degree video is a privacy nightmare, which can be overcome with planning and awareness. The issue is that everyone in any given area gets filmed.
  • You need a selfie stick! With 360 video you need a way of carrying it around without your face and body getting in the way. Carrying it at a consistent height is also an issue easily solved by dangling a piece of string to the floor. Who would have thought I would be involved in an innovation project with selfie sticks!
  • Work with your IT department. You need a place to store and manage large video (and other) files. Make sure the computers have the specs you need. At present we are hoping RMIT turns on YouTube as part of our Google Apps in Education, for its 360-degree hosting and streaming capability.
  • Work with your marketing department. They may have their own interest in the photo and video media or 3D objects you create, as well as access to potential resources.
  • In general work across your organisation rather than in your silo. We involved teaching staff and students across four schools within the College of Design and Social Context, IT and marketing as mentioned. It can be like herding cats to get diversely placed people into the one conversation but well worth it when you get there, for the interest and solutions that can arise.
  • One thing we didn’t learn – how to edit 360-degree video without paying someone a lot of money. No doubt we will address this in the future. Let us know if you have any suggestions!

This blog post is published under a Creative Commons Attribution (cc-by) licence.

Collaborative Mapping

I just discovered GroupMap, an online tool for collaborative mind-mapping that works across multiple devices – a simple idea that, if well executed, could be a winner. The app is browser based and works on Chrome, Safari and Safari for iPad.

A mind-mapping diagram

A map created with GroupMap

I have been trying to work out possible applications and the immediate thing that comes to mind is that it could be used in conjunction with a video conferencing tool such as Collaborate, as a more lasting and effective activity than the whiteboard.

"So all we need to do is build an app and share it?"

Four children sitting around a table, interacting with an iPad

One of the challenges I have found in supporting e-learning projects is to figure out how to get them to complete and at the same time how to get the best shareable benefits for future e-learning activities. Projects done for the former Framework or the current VET E-learning Strategy are required to return and share their outcomes. This is so that other training providers can benefit from their findings and/or content and to save the wheel from continually being reinvented.

At the end of the recent induction morning for the E-learning for Participation and Skills funding, one of the participants comes to me with a question. He is a techie and good at his job and his question is “So all we need to do is build an app and share it?” He wants to make sure he meets all the project outcomes requirements. I said yes, but I now realise this answer needs qualification.

Sometimes apps get built that are organisation specific, in that they are integrated to the organisation’s student management system or LMS. When it comes to the end of the project the shareable app may exist but only in a form that integrates only with their system. They can make the code available but it has to be reworked and as such may not be all that helpful to other organisations. They may as well start again. (This is a similar issue to LMS outputs in a packaged form that can be shared vs. in the form of a whole course, as a sophisticated design does not easily package.)

For mobile projects it brings up the issue of designing for mobile: should organisations create an organisational-based app on which all their other apps are custom built, providing easy access and functionality, or should they customise generic apps that anyone can use and that do not reference any database or LMS of the original creator?

On one hand, the latter is better in terms of shareable project outcomes. On the other hand, the former is better for model development outcomes where the benefit is the story of how they went about embedding the new e-learning development activity. I am not saying either is better – in the past I think we have gotten too caught up in sharing content which leads to some of the shareability problems – when in fact the best outcome is the sharing of the development model in the context of their organisation. What matters is that projects need a clear goal at the start as to what they are offering and delivering.

Delivering both shareable content and learnings is rare. Usually project teams, often new to e-learning or fixed in old ways of thinking, can only deliver either a poorly developed learning resource or a model report. The best win-win for project outcomes is when a team delivers a learning resource (ie. a packaged learning object or generic app) that is highly shareable (rather than just technically shareable) and facillates sharing of the organisational model developed in the process.

Photo: NASA Visualization Explorer (iPad app), by NASA Goddard Photo and Video on flickr.

conVerge12 and Twitter

A presenter in front of her audience

Nancy White at conVerge11

I was asked recently to provide a list of tools for displaying and collecting Twitter feeds for our next e-learning conference, conVerge12.

The wonderful thing about using Twitter for a conference is that it extends the conference beyond the time and space of its various presentations. Outsiders who can’t make it can get a sense of what’s going on in the conversations. Tweeting of keynotes at conVerge11 has helped improve the reputation of the conference as others in Australia got a taste of what was happening in real time.

Another great advantage of using Twitter is that it can enrich presentations: questions to the speaker can be put through the feed, and speakers can sometimes ask the audience to insert their own content into the feed (such as pictures).

So a few tools are available for displaying tweets on a big screen, along side the presenter’s PowerPoint or live video feed. I decided to go for Twitterwall, as it’s free and works great, but there is also Tweetwall.

What makes these tools especially cool and powerful is that they allow tweets to be validated before they appear on the screen. This is very interesting as it prevents spamming, and it allows the administrator (me) to exclude irrelevant and non-suitable tweets (ie. tweets that contain aggressive or disrespectful language for instance).

If you want to have a more attractive display of the tweets, you can use Visible Tweets. I’ve used it during presentation breaks in the break areas to give reminders of what was said.

After an event you may want to have a record of all the tweets. Last year I used tweetdoc which saves all the tweets with the conference hashtag into a PDF document. I also like the look of If This Then That for directing a set of hashtags into the cloud service Evernote.

Photo: Nancy White, by howard61 on flickr.