Flemish 3D image processing technology brings inaccessible Roman catacombs to life in incredible detail (and over 120 times faster)

Pioneering technology by iMinds - KU Leuven has a wide range of potential applications, from opening up cultural heritage to inspecting nuclear power stations and accurate positioning of self-driving cars.

Background

Autonomous or self-driving robots are increasingly being used to map certain locations in detail, especially when those places are difficult for humans to access (narrow catacombs and sewers, for example) or when they entail safety or health risks (consider the internal inspection of a nuclear power plant following a catastrophe). That was the starting point for the European ROVINA project that was recently brought to a successful conclusion.

“But in practice, these robots still face a lot of problems,” says Professor Luc Van Gool (iMinds - KU Leuven). “For example, the use of traditional imaging techniques such as HD video is often impossible because the robots are (literally) stumbling around in poorly lit areas – all while laser technology is still very expensive."

That is why, in the context of the ROVINA project, the iMinds - KU Leuven researchers developed new, affordable technologies to create good-quality images; an approach that works even in difficult circumstances (such as the dark Catacombs of Priscilla in Rome). They combined this with 3D reconstruction software that can compile pictures into lifelike, realistically colored 3D images.

Research results

To be able to take good-quality photographs in the dark catacombs, the Flemish scientists developed an array of seven cameras and a special light configuration for the self-driving ROVINA robot.

“The fact that we were able to collect good source material that can be used by our 3D image reconstruction software was a first important breakthrough. But even more significantly, we have also been able to improve our intelligent image processing algorithms. As such, we can now realistically construct large-scale 3D models, based on a gigantic amount of data,” Professor Van Gool explains. “What is particularly striking is the amount of detail we can get out of our 3D models: in the case of the Roman catacombs, it feels as if you are really wandering around in them ; and you can zoom in on the tiniest details, such as the lichen on the stones or the ancient cement work.”

“Moreover, our algorithms are over 120 times faster than existing methods, which is necessary to turn large quantities of data and images into a lifelike 3D model within a reasonable time frame,” adds researcher Marc Proesmans (iMinds - KU Leuven). “What used to take more than 120 days can now be done in just one day. That means that these applications can now also be introduced beyond the academic world.”

Tests with existing algorithms indeed showed that it used to take three to four days to generate 3D images based on recordings from a fifteen-yard passageway. However, that makes this technology unusable for projects such as ROVINA, where more than six hundred yards of passages needed to be mapped.

Future perspectives

Opening up cultural heritage that is under threat (or difficult to access) is just one possible application.

“We can actually use our findings in any situation where we want to accurately replicate – in 3D – what a robot ‘sees’. Gaming is one example. But this is also an important step forward in the development of self-driving cars. After all, a self-driving car needs to be able to estimate accurately, and in real time, where it is located. GPS – which has an accuracy of a few yards – is not sufficient for this. But our technology is: the car ‘observes’ its surroundings and uses the resulting data to locate its position with an accuracy of a couple of inches. As such, we are inching closer to a world in which fully autonomous cars are a reality,” Professor Van Gool concludes.