AI-based 3D reconstruction from a hand-held visual-laser system

Digitising the geometry of existing infrastructure assets is a key element for many use cases in the Architecture, Engineering and Construction (AEC) industries. The latter is among the least digitised compared to all other sectors of the economy. It is estimated that improving this situation could result in higher productivity which, in turn, could create up to $1.6 trillion of additional value-added, meeting half of the world's infrastructure needs. One of the key elements of digitisation in this context is mapping, i.e. creating pointclouds of an environment using such sensors as a camera or a lidar.

Use cases such as engineering surveying, measured surveys of infrastructure scenes and modelling are regarded as the core based on which subsequent phases of planning, design and development are based in the AEC industry. They have certain performance requirements, the crucial of which are the accuracy/noise of the created pointclouds, the distance from which the scene can be mapped, the density of the pointclouds so that they are informative enough for the end-user and the ease of use of the mapping device.

Given these performance requirements, one of the identified knowledge gaps is that we do not know how to accomplish middle-distance (10 - 30 meters) mobile mapping while maintaining acceptable density and accuracy of the scanned objects in near real-time. None of the existing technologies can achieve that.

Accordingly, a prototype of a mobile scanning system optimised explicitly for infrastructure scenes has been devised in this research conducted by PhD candidate Maciej Trzeciak. Specifically, a visual-laser system based on the combination of deep learning-based image-guided depth completion and visual-lidar odometry/SLAM is proposed. The former will densify the lidar range measurements up to the level required at mid-range distances while the latter will provide the current state-of-the-art trajectory estimation crucial for georeferencing the densified measurements. In other words, this framework will fuse the best features of high-resolution cameras and low-resolution lidars in mobile mapping.

Initial results suggest that the system exceeds the performance of currently available technologies for indoor scenes (a 3D reconstruction of a cupboard is shown in the figure), and further research will be carried out to investigate the proposed framework for outdoor middle-distance infrastructure scenes.