Augmented reality: soon to be the new normal on navy ships

With an enthusiastic team of specialists, Björn Mes is working hard to use Augmented Reality (AR) technology to create a successful product. To that end, the AR and VR Research Engineer is working closely with the people from P&P, who are involved in product development at Damen Schelde Naval Shipbuilding.

Looking through a pair of AR glasses, you will see the real world with an artificial image superimposed over it. “Reality is enriched; that is the difference with Virtual Reality (VR), where you are locked in a completely artificial world. Everything you see is fake,” says Björn Mes. “The AR glasses add data from other, mostly digital sources, to what you see. This gives the data a physical context, becoming information that is of use to you. AR can be put to good use a naval vessel. Naval ships are processing more and more data through systems for combat management, platform management, the bridge and maintenance. All of these systems are automated. At the same time, the trend is that crews are reducing in number. AR technology can help crews access and process all that data in a smarter way. Imagine that you are the officer on duty on the bridge – AR ensures that you get exactly the data you need at that moment.”

According to Björn, it is a challenge to refine AR technology in such a way that it ‘recognizes’ each individual naval ship. “On these ‘grey’ ships you see a quite a lot of standardisation; many things on board are similar,” he says. “The AR glasses find it difficult to recognise specific objects; they have to adjust to the ship’s environment. The fact that the ships are made of steel and use radio signals can have a disruptive effect and therefore hinder the optimal functioning of AR.”

To address this, Björn’s team is experimenting with Microsoft’s Hololens 2. These are AR glasses that utilise external sources such as multiple cameras, internal gyroscopes and compasses to slide an overlay over the real world that we see through the glasses. “The Hololens 2 translates its environment into a 3D model and compares that with the 3D model that is stored in the computer program with which we design ships. But it also translates objects into a 3D model, so you can see their depth of field. If you talk to it, the Hololens 2 uses and ‘translates’ your input to interact with both the 3D model it has created and the digital world to which the 3D model refers. It can even turn your hands – if it recognises them – into a 3D object that you can then use as an instrument in the digital world.”

AR allows you to see a pipe behind a wall, which is impossible to observe in real life. You can also see in no time where the pipe was manufactured and whether it is damaged. “There will come a time when AR is the most natural thing in the world for everyone,” expects Björn. “Our environment is becoming increasingly digital, and to process those sources you need tools that involve AR.” In any case, DSNS is already experimenting with it. “Thales – for the sensors – and our trusted automation company RH Marine can also play an important role in practical applications.”