DNS SLAM Dense Neural Semantic-Informed SLAM

DNS SLAM Dense Neural Semantic-Informed SLAM

IROS 2024

This repository contains the code for the paper DNS-SLAM, a neural semantic SLAM method that perform real-time camera tracking and dense reconstruction based on a joint encoding.

NOTE: This is a pre-released version, code is not fully ready yet.

Update

• [x] Code for DNS-SLAM [2024-8-20]

Installation

Please follow the instructions below to install the repo and dependencies.

git clone ...
cd dns-slam

Install the environment

# Create conda environment
conda create -n dns-slam python=3.7
conda activate dns-slam

# Install the pytorch first (Please check the cuda version)
pip install torch==1.10.1+cu113 torchvision==0.11.2+cu113 torchaudio==0.10.1 -f https://download.pytorch.org/whl/cu113/torch_stable.html

# Install all the dependencies via pip (Note here pytorch3d and tinycudann requires ~10min to build)
pip install -r requirements.txt

For tinycudann, if you cannot access network when you use GPUs, you can also try build from source as below:

# Build tinycudann 
git clone --recursive https://github.com/nvlabs/tiny-cuda-nn

# Try this version if you cannot use the latest version of tinycudann
#git reset --hard 91ee479d275d322a65726435040fc20b56b9c991
cd tiny-cuda-nn/bindings/torch
python setup.py install

Dataset

Replica

Download the sequences of the Replica Dataset generated by the authors of vMAP into your dataset folder.

ScanNet

Please follow the procedure on ScanNet website, and extract color & depth frames from the .sens file using the code.

Run

You can run DNS-SLAM using the code below:

# replica
python run.py configs/replica/room_0.yaml --input /mnt/user/datasets #replace as your root data path

#scannet
python run.py configs/scannet/scene0000.yaml --input /mnt/user/datasets #replace as your root data path

You can also change input(dataset_dir) and output(out_dir) path in configs/slam.yaml

Evaluation

You can run trajectory evaluation using the code below:

# replica
python eval_ate.py configs/replica/office_0.yaml

#scannet
python eval_ate.py configs/scannet/scene0000.yaml

You can run reconstruction evaluation using the code below:

# replica

#scannet

You can run visulation using the code below:

# replica
python visualizer.py configs/replica/office_0.yaml

#scannet
python visualizer.py configs/scannet/scene0000.yaml

Acknowledgement

Citation

If you find our code or paper useful for your research, please consider citing: