A well known travel brand looking to attract a new segment of customers asked to explore using VR to create an experience that would bend minds. The creative idea that we developed played off of one of the most compelling elements of recent virtual games — mixed reality. By introducing real physical objects, that can be touched and tracked in the virtual space, your grasp on reality begins to slip away.
Our first prototype played on this illusion in a similar way to the experience the lead actress in the movie Tomorrowland has when she touched a magical pin. We too hoped to design an experience that was so compelling, that when people experienced it, they would pull their hand off the objects of excitement and only re-engage once they were ready. Below is a reference video we showed the client.
Nothing existed through current commercial offerings that would allow us to have the instantaneous trigger and to track objects other than game controllers. Our solution was to create an "in game" calibration routine where the HTC Vive controller calibrated the location of the objects to be touched.
Using the Leap motion controller, a depth camera with a high fidelity of hand tracking you were able to view the distance between your hands and the objects you were going to touch.
We then configured a small linux computer (Raspberry Pi) to detect changes in capacitance of our objects. Objects that weren't naturally conductive were treated with capacitive spray paint. The python application running on the Raspberry Pi fired off UDP signals to our desktop running the Unity game engine.
"His recent work researching and prototyping a VR + physical computing concept was absolutely instrumental in bringing the idea to life for our internal and client teams in a way that is impossible to do on paper."
Christian Madden / SVP Creative Director, Technology at MullenLowe
"Brian conceived of and prototyped a VR experience that reimagined the human/machine interface. When we presented the idea on paper, the clients were intrigued. When a few weeks later they experienced Brian's prototype, they immediately gave the green light."
John Kearse / SVP Creative Director at Mullen