Extended Reality Tools Enhance Training

FOUR INDUSTRIAL REVOLUTIONS

To provide some context, Gentry noted what distinguished the industrial revolutions.

• The first, which occurred in the decades around 1800, marked the transition from human or animal work to water and steam power as the source of energy for manufacturing.
• In the second industrial revolution, water and steam power gave way to electricity as the source of energy, starting in the late 1800s. Other changes included the division of labor and development of assembly lines that increased productivity.
• In the third, electronics and information technology made automation widespread in the last half of the 1900s.
• Now, in the 21st century, the fourth industrial revolution, known as Industry 4.0, realizes the potential of networked and information technologies such as the Internet of Things, artificial intelligence, machine learning, augmented and virtual reality, and big data.

DIGITAL TWINS

Another aspect of Industry 4.0 is the use of digital twins. A virtual model of a real-world asset or process—its digital twin—can act as a bridge between the physical and digital worlds. You can try out changes on the virtual twin, evaluating them before making any changes on the real-world version. A twin also allows for data analysis and system monitoring that can help prevent downtime.

A virtual model can also be useful in training. “If I can make an exact digital replica of a machine, and put it in a virtual environment, a service technician can go in and take things apart and diagnose things,” Gentry said. This can be safer than working on the real thing, allows repeating the same operations and prevents damage to the real machine.

EXTENDED REALITY

Extended reality is an umbrella term that encompasses all immersive digital technologies. It includes the following:

• Augmented Reality adds images or data to a view of the real world. A simple example is a translation app. Aim your phone’s camera at written words and the app shows the translation. You can use a smartphone or a tablet for augmented reality experiences or a head-mounted device such as a voice-operated Android unit with a small display positioned in front of the user’s eyes.
• Virtual Reality places the user in the midst of a created space and requires a closed viewer so users cannot see their actual surroundings.
• Mixed Reality takes augment- ed reality further. It allows you to manipulate digital images as if they actually existed in the real world. Certain head-mounted devices that track hand and eye movements are designed for this so the user can move and interact with computer-generated images as if they were real.

TRAINING IN A VIRTUAL SPACE

Virtual reality can be useful in a classroom context. However, since the viewer obscures all of the actual surroundings, it would be too dangerous to use on the shop floor.

Augmented reality offers some useful abilities, such as popping up instructions on request.

Mixed reality, however, is well-adapted to training and real- time assistance. The software scans the environment, creating for itself spatial anchors to mark the locations of real-world objects. The user can attach computer images to the objects.

MIXED REALITY IN THE FIELD

Mazak has been using mixed-reality software to train field service engineers, aid them as they perform maintenance and give them expert help when needed. The field engineer wears a head-mounted display specifically designed for mixed reality.

In training mode and for normal maintenance, the software guides the engineer what to do, step by step. For example, when there is a button to push, a hand icon appears over the button on the console. If written instructions are needed, they appear on request.

If a situation occurs that is beyond the engineer’s current knowledge, a quick request for assistance brings up an expert on video, who can be present virtually to walk through troubleshooting and repair.

STEP BY STEP

During his presentation at the Valve Forum, Gentry showed a video of the software tools Mazak is using, along with comments from staff about the value of these training and assistance options.

“With [this display and software] you can call me with a problem, or a customer can call me with a problem,” said Ron Kelley, Mazak maintenance training instructor, “and I can walk them through the whole problem by seeing what’s going on.”

The mixed-reality guide and remote assistance software “absolutely have improved productivity. I can send out a younger engineer who’s very smart and very experienced but not with Mazak machines,” Kelley said, “and it’s almost like I send him with an experienced person. It’s a total game changer. Otherwise, we’d have to pair him up with a seasoned Mazak engineer for four, five, maybe six months to gain all that experience.”

For the past 15 or 20 years, we’ve been wondering how to fill the skills gap left by Baby Boomer workers retiring, Gentry said. “We’re dealing with how do we take all of that knowledge and give it to the individuals coming up now. The key here is how do we develop the new generation to utilize the new technology and take the knowledge from the retiring generation and apply it? We really need both.”

Barbara Donohue is web editor at VALVE Magazine.