Gaussian-LIC: Real-Time Photo-Realistic SLAM with Gaussian Splatting and LiDAR-Inertial-Camera Fusion

Xiaolei Lang1, Laijian Li1, Chenming Wu2, Chen Zhao2, Lina Liu1, Yong Liu1, Jiajun Lv1, Xingxing Zuo3
1Zhejiang University, 2Baidu VIS, 3Technical University of Munich
Paper arXiv Code
Image 1
Image 2
Image 3

Abstract

In this paper, we present a real-time photo-realistic SLAM method based on marrying Gaussian Splatting with LiDAR-Inertial-Camera SLAM. Most existing radiance-field-based SLAM systems mainly focus on bounded indoor environments, equipped with RGB-D or RGB sensors. However, they are prone to decline when expanding to unbounded scenes or encountering adverse conditions, such as violent motions and changing illumination. In contrast, oriented to general scenarios, our approach additionally tightly fuses LiDAR, IMU, and camera for robust pose estimation and photo-realistic online mapping. To compensate for regions unobserved by the LiDAR, we propose to integrate both the triangulated visual points from images and LiDAR points for initializing 3D Gaussians. In addition, the modeling of the sky and varying camera exposure have been realized for high-quality rendering. Notably, we implement our system purely with C++ and CUDA, and meticulously design a series of strategies to accelerate the online optimization of the Gaussian-based scene representation. Extensive experiments demonstrate that our method outperforms its counterparts while maintaining real-time capability. Impressively, regarding photo-realistic mapping, our method with our estimated poses even surpasses all compared methods that utilize privileged ground-truth poses for mapping.

Method

Gaussian-LIC consists of a tracking module and a mapping module powered by the carefully designed acceleration strategies for real-time performance. The former tightly couples multimodal sensory data to deliver robust and accurate pose estimation, outputting geometrically precise colorized LiDAR points with triangulated SFM points for compensation. The latter receives the sequential data from the former and incrementally reconstructs a Gaussian map, where the sky and the varying camera exposure are both properly modeled for better visual quality.

Experimental Results

Qualitative rendering performance. Now that all compared methods fail to estimate accurate poses in complex unbounded scenarios, we run them with ground truth poses so as to fairly compare the photo-realistic mapping performance.

BibTeX

@article{lang2024gaussian,
  author={Lang, Xiaolei and Li, Laijian and Wu, Chenming and Zhao, Chen and Liu, Lina and Liu, Yong and Lv, Jiajun and Zuo, Xingxing},
  journal={arXiv preprint arXiv:2404.06926}, 
  title={Gaussian-LIC: Real-Time Photo-Realistic SLAM with Gaussian Splatting and LiDAR-Inertial-Camera Fusion}, 
  year={2024}
}