Augmented Instructions

Facilitating lenses. Overlaying for simplicity.

Applying the Microsoft HoloLens 2 to deliver real-time training; cutting out errors and speeding up employee onboarding.

If your only experience with AR or VR is having seen people in goggles playing games, you’d be forgiven for wondering how that tech they could be used in an industry like life sciences. However, the adoption of AR and VR is growing. One example of how we do it at Novo Nordisk is by using augmented reality technologies to streamline onboarding and enhance performance across departments.

But there is even more potential to be unlocked.

Applying these technologies represents a big shift that can go beyond simply training and reducing costs, such as creating new digital ways of designing the next facility or equipment – and optimising processes at production facilities.

The way it works is by supporting with instructions: for example, a how-to-repair-manual is designed, created and “fed” into the HoloLens software. Then, when wearing the HoloLens 2 glasses, the user can perceive a holographic layer which is superimposed over their actual surroundings or, a machine.

This layer can then present step-by-step job instructions in versatile formats such as text, photos, videos, and 3D elements. This blending of the physical and virtual worlds facilitates an immersive experience, that guides employees through their tasks seamlessly via the device. The user can navigate complex procedures in real-time, ensuring the accurate completion of every step while gaining an interactive visualisation of their work environment.

So, for example, a new worker at the factory is confronted with a 20ft conveyor belt that requires maintenance. The usual process would be finding a trainer – or waiting until one becomes available – or finding the Standard Operating Procedure, and then sifting through it all to work out how to fix the machine. With the HoloLens 2, that worker can now see the machine in real time – hands free – and the overlay instructions can guide the user on how best to perform a specific process such as preparing the line for operations.

The introduction of augmented instructions has significantly streamlined the onboarding process for employees in Tianjin, and it also lets trainees practice tasks autonomously which leads to a noticeable proficiency improvement and a significant increase of training RFT.

Moreover, trainer time has been reduced substantially, allowing them to use their time on other crucial tasks. The HoloLens 2 also enables employees to engage in a wider range of activities while seamlessly sharing their real-time perspective with offsite colleagues through the Remote Assist function.

The tech behind the tech

It’s probably no surprise that creating this kind of mixed reality process involves complicated tech. The core solution involves a range of Microsoft technologies, including HoloLens 2, Microsoft Dynamics 365, Microsoft Dynamics 365 Guides (including Remote Assist), and Microsoft Power Platform.