search

VIS4ROB-lab / covins

COVINS-(G) -- A (Generic) Framework for Collaborative Visual-Inertial SLAM and Multi-Agent 3D Mapping
394 stars 64 forks source link

readme

COVINS-(G) -- A (Generic) Framework for Collaborative Visual-Inertial SLAM and Multi-Agent 3D Mapping

Version 2.0 (Support for COVINS-G: A Generic Back-end for Collaborative Visual-Inertial SLAM)

COVINS is an accurate, scalable, and versatile visual-inertial collaborative SLAM system, that enables a group of agents to simultaneously co-localize and jointly map an environment. COVINS provides a server back-end for collaborative SLAM, running on a local machine or a remote cloud instance, generating collaborative estimates from map data contributed by different agents running Visual-Inertial Odomety (VIO) and sharing their map with the back-end.

With the COVINS-G release, we make the server back-end flexible enabling compatibility with any arbitrary VIO/Stereo front-end, including, for example, off-the-shelf cameras with odometry capabilities, such as the Realsense T265.

We provide guidance and examples how to run COVINS and COVINS-G on the EuRoC dataset, as well as information beyond basic deployment, for example how the COVINS back-end can be deployed on a remote cloud computing instance. Instructions on running COVINS-G with different cameras such as Intel Realsense D455 and T265 Tracking camera, as well as different frontends like VINS-Fusion, ORB-SLAM3 and SVO-pro are also provided.

Index

1 Related Publications

[COVINS-G] Manthan Patel, Marco Karrer, Philipp Bänninger and Margarita Chli. COVINS-G: A Generic Back-end for Collaborative Visual-Inertial SLAM. IEEE International Conference on Robotics and Automation (ICRA), 2023. PDF

[COVINS] Patrik Schmuck, Thomas Ziegler, Marco Karrer, Jonathan Perraudin and Margarita Chli. COVINS: Visual-Inertial SLAM for Centralized Collaboration. IEEE International Symposium on Mixed and Augmented Reality (ISMAR), 2021. PDF

[Redundancy Detection] Patrik Schmuck and Margarita Chli. On the Redundancy Detection in Keyframe-based SLAM. IEEE International Conference on 3D Vision (3DV), 2019. PDF.

[System Architecture] Patrik Schmuck and Margarita Chli. CCM‐SLAM: Robust and Efficient Centralized Collaborative Monocular Simultaneous Localization and Mapping for Robotic Teams. Journal of Field Robotics (JFR), 2019. PDF

[Collaborative VI-SLAM] Patrik Schmuck, Marco Karrer and Margarita Chli. CVI-SLAM - Collaborative Visual-Inertial SLAM. IEEE Robotics and Automation Letters (RA-L), 2018. PDF

Video:

COVINS-G: COVINS:
Mesh Mesh

2 License

COVINS is released under a GPLv3 license. For a list of code/library dependencies (and associated licenses), please see thirdparty_code.md.

For license-related questions, please contact the authors: collaborative (dot) slam (at) gmail (dot) com.

If you use COVINS in an academic work, please cite:

@article{schmuck2021covins,
  title={COVINS: Visual-Inertial SLAM for Centralized Collaboration},
  author={Schmuck, Patrik and Ziegler, Thomas and Karrer, Marco and Perraudin, Jonathan and Chli, Margarita},
  journal={arXiv preprint arXiv:2108.05756},
  year={2021}
}

If you use the COVINS-G extension in an academic work, please cite both the original COVINS publication and additionally:

@article{patel23covinsg,
  title = {COVINS-G: A Generic Back-end for Collaborative Visual-Inertial SLAM},
  doi = {10.48550/ARXIV.2301.07147},
  url = {https://arxiv.org/abs/2301.07147},
  author = {Patel, Manthan and Karrer, Marco and Bänninger, Philipp and Chli, Margarita},
  publisher = {arXiv},
  year = {2023},
}

3 Basic Setup

This section explains how you can build the COVINS server back-end, as well as the provided version of the ORB-SLAM3 front-end able to communicate with the back-end. Morover, the COVINS-G extension provides a generic ROS front-end wrapper to support any arbitrary VIO/tracking cameras as well as support for VINS-Fusion. We provide installation instructions for Ubuntu 18.04 as well as 20.04. Note that we also provide a Docker implementation for simplified deployment of COVINS.

Note: Please pay attention to the CMAKE_BUILD_TYPE. Particularly, building parts of the code with march=native can cause problems on some machines.

Environment Setup

Dependencies

Set up your workspace

This will create a workspace for COVINS as ~/ws/covins_ws. All further commands will use this path structure - if you decide to change the workspace path, you will need to adjust the commands accordingly.

COVINS Installation

Note on Virtual Machines

If you are installing COVINS in a Virtual Machine (VM) or a computer with fewer resources, make sure to allocate enough memory and/or swap to ensure a successful installation. We have successfully tested the COVINS installation with a VM with 16 GB RAM and 10 GB swap. Instructions how to change the swap size can be found here.

Installation Manual

We provide a script (covins/install_file.sh) that will perform a full installation of COVINS, including back-end, front-end, and third-party packages, if the environment is set up correctly. If the installation fails, we strongly recommend executing the steps in the build script manually one by one. The script might not perform a correct installation under certain circumstances if executed multiple times.

Generally, when the build process of COVINS or ORB-SLAM3 fails, make sure you have correctly sourced the workspace, and that the libraries in the third-party folders, such as DBoW2 and g2o are built correctly.

A remark on fix_eigen_deps.sh: compiling code with dependencies against multiple Eigen versions is usually fatal and must be avoided. Therefore, we specify and download the Eigen version explicitly through the eigen_catkin package, and make sure all Eigen dependencies point to this package.

Installing ROS Support for the ORB-SLAM3 Front-End

If you want to use rosbag files to pass sensor data to COVINS, you need to explicitly build the ORB-SLAM3 front-end with ROS support.

Installing VINS-Fusion Front-End

If you would like to use a VINS-Fusion Front-end (only compatible with COVINS-G), we provide a modified version of the original VINS-Fusion repository which is made compatible with COVINS. The modified repository is available here (VINS-COVINS-adaptation). Follow the Readme for installation instructions. Note, we only provide support for using VINS-Fusion with docker due to compilation issue of VINS-Fusion on Ubuntu 20.04.

4 Running COVINS

We provide easy switching between the different backends (COVINS and COVINS_G) by specifying the placerec.type parameter in the ~/ws/covins_ws/src/covins/covins_backend/config/config_backend.yaml file. In general, COVINS would be able to provide more accurate estimates after performing the Global Bundle Adjustment (GBA), but is limited by the flexibility in terms of the choice of front-end (works well with ORB_SLAM3 but not VINS-Fusion). On the other hand, COVINS_G is compatible with any arbitrary front-end but cannot perform GBA since map-points are not utilized in the back-end. We refer to the COVINS_G paper for more details about the use-cases of the different back-ends.

COVINS Back-End

The instructions for running a COVINS back-end can be found here.

COVINS-G Back-End

The instructions for running a COVINS-G back-end can be found here.

5 Docker Implementation

We provide COVINS also as a Docker implementation. A guide how to install docker can be found here. To avoid the need of sudo when running the commands below you can add your user to the docker group.

sudo usermod -aG docker $USER (see https://docs.docker.com/engine/install/linux-postinstall/)

Building the Docker Image

NOTE: The provided docker implementation will not build if you have previously built your cloned version of COVINS in a catkin workspace using the instructions provided above. If you want to use the docker implementation, please perform the docker build with a freshly cloned version of COVINS

Build the docker file using the Make file provided in the docker folder. Provide the number of jobs make and catkin build should use. This can take a while. If the build fails try again with a reduced number of jobs value.

Running the Docker Image

The docker image can be used to run different parts of COVINS (e.g. server, ORB-SLAM3 front-end, ...).

ROSCORE

To start the roscore one can either use the host system ROS implementation (if ROS is installed). Otherwise, it can be started using the docker image.

COVINS Server Back-End

The convins server back-end needs a running roscore, how to start one see above. Furthermore, the server needs two configuration files, one for the communication server on one for the back-end. These two files need to be linked when running the docker image.

ORB-SLAM3 Front-End

The ORB-SLAM3 front-end client needs the communication server config file, the file which should be executed, and the path to the dataset. The dataset has to be given seperately since the file system of the docker container differs from the host system. Hence, the pathDatasetEuroc variable in the run script gets adapted automatically inside the docker container.

ORB-SLAM3 ROS Front-End

The ROS wrapper of the ORB-SLAM3 front-end can also be started in the docker container. It requires the server config file and the ROS launch file. A bag file can then for example be played on the host system.

ROS Front-End Wrapper (Only for COVINS-G)

The ROS Front-end wrapper can also be started in the docker container. It requires the server config file and the ROS launch file. A bag file can then for example be played on the host system.

Terminal

A terminal within the docker image can also be opened. This can for example be used to send rosservice commands.

6 Extended Functionalities

Deployment of the COVINS Back-End in an AWS Cloud Instance

We have tested COVINS with remote cloud servers provides by Amazons AWS. We recommend using screen to run COVINS remotely on a cloud instance, since this allows to return to previous sessions evens after disconnecting from the remote instance. Tutorial: Get started with Amazon EC2 Linux instances

Cloud Setup