With Mixed Reality, the Expert is Always ‘At Hand’

Mixed reality is a variation of virtual reality that merges real and virtual worlds where physical and digital objects co-exist and interact in real time.  Mixed reality systems often include the use of a head-mounted display with a small screen in front of the eyes that generates realistic images, sounds and other sensations projected on top of the physical environment that a user can interact with.  Mixed and virtual reality applications first appeared in video games and subsequently in medical and military training applications.  While still labeled as an advanced technology, there is a considerable opportunity to gain benefits from mixed reality in public transit today. 

In 2019, MV Transportation began collaborating with their first customers interested in deploying electric vehicles (EV) as part of their transit fleets.  Those efforts soon led to a small but growing number of new battery-powered buses for MV to operate and maintain, which in turn drove training and development efforts for our maintenance organization.  While MV has decades of experience operating gasoline, diesel and CNG vehicles, electric vehicles have introduced new factors related to battery maintenance, charging strategies and route planning that required upskilling our professional ranks. 

With a limited number of EV subject matter experts in the field, we turned to mixed reality – in the form of Microsoft’s second-generation Hololens device and application environment – to enable these professionals to connect with one another.  With the local technician wearing the device, a subject matter expert across the country can “see” through their eyes, helping identify components or walk them through complex tasks.  In some instances, the subject matter expert was an OEM bus technician connecting remotely through their PC.   This ability to remotely connect became critically important during the COVID-19 pandemic when business travel was simply not possible. 

After a successful pilot in support of our electric vehicle fleets, we expanded use of mixed reality to enable an enriched training environment that has proven to be far more effective than traditional methods.  In these scenarios, the technician can see a virtual image of a particular part or reference pages of a virtual instruction manual while they are working on their repair or preventative maintenance task.  This effort alone has greatly improved productivity and reduced the time and learning curve of our maintenance professionals, which remain in critically high demand across the industry. 

MV’s early results with mixed reality reveal the potential to eliminate hours and even days of potential vehicle down time on complex repairs by connecting with a subject matter expert or accessing manufacturer’s instructions immediately.  Further, we anticipate the technology will help accelerate our team’s deployment of electric vehicles while expertise in the field remains limited.

Mixed reality should now be considered a mature technology with costs continuing to come down, making it an attractive and beneficial new tool for the transit toolbox.  With just one device and a strong network, a team can benefit from expertise across town, the country or the world.