Gaussian Splatting SLAM
*Hidenobu Matsuki · *Riku Murai · Paul H.J. Kelly · Andrew J. Davison
(* Equal Contribution)
CVPR 2024 (Highlight)
[comment]: <> (PAPER
)Paper | Video | Project Page
This software implements dense SLAM system presented in our paper Gaussian Splatting SLAM in CVPR'24. The method demonstrates the first monocular SLAM solely based on 3D Gaussian Splatting (left), which also supports Stereo/RGB-D inputs (middle/right).
# Note - In an academic paper, please refer to our work as **Gaussian Splatting SLAM** or **MonoGS** for short (this repo's name) to avoid confusion with other works. - Differential Gaussian Rasteriser with camera pose gradient computation is available [here](https://github.com/rmurai0610/diff-gaussian-rasterization-w-pose.git). - **[New]** Speed-up version of our code is available in `dev.speedup` branch, It achieves up to 10fps on monocular fr3/office sequence while keeping consistent performance (tested on RTX4090/i9-12900K). The code will be merged into the main branch after further refactoring and testing. # Getting Started ## Installation ``` git clone https://github.com/muskie82/MonoGS.git --recursive cd MonoGS ``` Setup the environment. ``` conda env create -f environment.yml conda activate MonoGS ``` Depending on your setup, please change the dependency version of pytorch/cudatoolkit in `environment.yml` by following [this document](https://pytorch.org/get-started/previous-versions/). Our test setup were: - Ubuntu 20.04: `pytorch==1.12.1 torchvision==0.13.1 torchaudio==0.12.1 cudatoolkit=11.6` - Ubuntu 18.04: `pytorch==1.12.1 torchvision==0.13.1 torchaudio==0.12.1 cudatoolkit=11.3` ## Quick Demo ``` bash scripts/download_tum.sh python slam.py --config configs/mono/tum/fr3_office.yaml ``` You will see a GUI window pops up. ## Downloading Datasets Running the following scripts will automatically download datasets to the `./datasets` folder. ### TUM-RGBD dataset ```bash bash scripts/download_tum.sh ``` ### Replica dataset ```bash bash scripts/download_replica.sh ``` ### EuRoC MAV dataset ```bash bash scripts/download_euroc.sh ``` ## Run ### Monocular ```bash python slam.py --config configs/mono/tum/fr3_office.yaml ``` ### RGB-D ```bash python slam.py --config configs/rgbd/tum/fr3_office.yaml ``` ```bash python slam.py --config configs/rgbd/replica/office0.yaml ``` Or the single process version as ```bash python slam.py --config configs/rgbd/replica/office0_sp.yaml ``` ### Stereo (experimental) ```bash python slam.py --config configs/stereo/euroc/mh02.yaml ``` ## Live demo with Realsense First, you'll need to install `pyrealsense2`. Inside the conda environment, run: ```bash pip install pyrealsense2 ``` Connect the realsense camera to the PC on a **USB-3** port and then run: ```bash python slam.py --config configs/live/realsense.yaml ``` We tested the method with [Intel Realsense d455](https://www.mouser.co.uk/new/intel/intel-realsense-depth-camera-d455/). We recommend using a similar global shutter camera for robust camera tracking. Please avoid aggressive camera motion, especially before the initial BA is performed. Check out [the first 15 seconds of our YouTube video](https://youtu.be/x604ghp9R_Q?si=S21HgeVTVfNe0BVL) to see how you should move the camera for initialisation. We recommend to use the code in `dev.speed-up` branch for live demo.
