When to open source COVINS-G

[UESTCgr]

Hi, I read your ICRA2023 paper "COVINS-G: A Generic Back-end for Collaborative Visual-Inertial SLAM". Great work! The paper shows that you will open source the code soon, I'm very interested in the job, thank you!

[manthan99]

Hi!

Yes, we are in the process of merging the repo to the codebase. Meanwhile, you can have a look at the following branch if you want to use it. While the installation instructions would be the same, it is still lacking the instructions/readme for running.

https://github.com/manthan99/covins/tree/vins-covins

Best Manthan

[luis-981102]

How to run it as a front end through VINS-Mono?

[manthan99]

Hi luis

The easiest way (without modifying the VINS codebase) could be to use the ROS frontend wrapper that we provide. Simply build the covins_frontend package where you will be running the VINS agent. You can then run your VINS-agent and simultaneously in another terminal run the first agent launch file. Make sure that the odometry argument of this launch file is modified to be the same as that where VINS would be publishing the odometry.

In any case, we would be releasing the codebase in a few days with proper instructions. Moroever, we will also be releasing the modified VINS-codebase with integrated COVINS communication so that you can directly use the keyframes created by VINS. With the frontend wrapper, we sample the image frames using some threshold distance and angle and don't utilize the keyframes so there might be a drop in accuracy while using the frontend wrapper.

Best Manthan

[yonggaogit]

@manthan99
Hi! COVINS-G is an excellent work. When using COVINS, I found that it does not support Stereo-Inertial very well. Does COVINS-G provide a Stereo-Inertial interface? I want to use the Stereo-Inertial mode of VINS-Fusion as the front end of COVINS-G. Is it possible? Thank you so much!

[manthan99]

Hi @yonggaogit

Yes, with COVINS-G you will be able to run any VIO, Stereo, or Stereo-Inertial frontend including the Stereo-Inertial mode of VINS-Fusion.

Best Manthan

[yonggaogit]

Hi @yonggaogit

Yes, with COVINS-G you will be able to run any VIO, Stereo, or Stereo-Inertial frontend including the Stereo-Inertial mode of VINS-Fusion.

Best Manthan

Thanks! I have tried COVINS-G multi-machine mode on two machines with realsense D435i installed and it works great. But I have a question, that is, the output topic of COVINS-G is used for visualization on rviz, but now I want to get the optimized multi-machine pose data for further processing. Is there a corresponding topic interface?

[manthan99]

Hi @yonggaogit

Thanks for the feedback and happy to know that it works for you.

Unfortunately, we do not have the support for accessing the optimized collaborative SLAM estimates at the moment on a ROS topic. Maybe #18 might be relevant for you.

Thank you Best Manthan

[UESTCgr]

Hi! I have tried COVINS-G on two agents and a server with ORB-SLAM3 front-end and worked fine.

But, I want to use a LiDAR Visual Inertial odometry to enhance the robustness of the system and further consturct a dense map (for example, using Fast-LIVO or R3LIVE as front-end) .

The COVINS-G paper showed that the back-end only needs KeyFrames and the corresponding poses ( I noticed that you have tested on a T265 camera and worked fine) , is there any possibility to use the odometry and image frames generated by a LIVO front-end in the COVINS-G based system?

[manthan99]

Hi @UESTCgr

Sorry for the late reply.

I am personally not familiar with the LIVO-frontend, but if it provides image frames with the odometry information, it should be possible to directly use it with COVINS-G using our ROS frontend wrapper. It should be noted that in this case, the backend doesn't receive any map points so you will have to run something else parallelly for getting the dense map (by using the optimized centralized KF poses)

I hope this helps.

Thank you Best Manthan

[manthan99]

Hi everyone!

You can now try out COVINS-G (#34)! We have added the running instructions for running with different front-ends including VINS-Fusion, ORBSLAM3, and SVO-Pro.

[NatureLan-sudo]

hi, @manthan99 , I have tried COVINS-G with ORB-SLAM3 front-end , use Mono_Inertial Node, it work fine. When I tried to compile the ORBSLAM3 stereo-IMU node and run with two agents, I encountered frequent map resets and initialization failures. I would like to know if the current version of the ORBSLAM3 frontend has been adapted for stereo-IMU versions. If not, how can I modify it to make the program work properly?

[manthan99]

Hi @NatureLan-sudo

May I ask which dataset are you using or are you using your own camera setup?

Regarding your question, we have not made any changes to the ORB-SLAM3 Front-end but rather modified the back-end so that it is able to work purely based on 2D keypoint information with the COVINS-G release. Similar to COVINS, we still are not able to handle map resets done by ORB_SLAM3.

If you are not constrained by the choice of front-end, I would recommend trying out with a VINS-mono Front-end in stereo-inertial mode. VINS seems to work more reliably than ORB_SLAM3 which does frequent map resets (our back-end is not able to handle this cases). You can find the instructions here: https://github.com/manthan99/covins/blob/vins-covins/docs/run_COVINS-G.md#run_fe_vins

Best Manthan

[NatureLan-sudo]

Hi @NatureLan-sudo

May I ask which dataset are you using or are you using your own camera setup?

Regarding your question, we have not made any changes to the ORB-SLAM3 Front-end but rather modified the back-end so that it is able to work purely based on 2D keypoint information with the COVINS-G release. Similar to COVINS, we still are not able to handle map resets done by ORB_SLAM3.

If you are not constrained by the choice of front-end, I would recommend trying out with a VINS-mono Front-end in stereo-inertial mode. VINS seems to work more reliably than ORB_SLAM3 which does frequent map resets (our back-end is not able to handle this cases). You can find the instructions here: https://github.com/manthan99/covins/blob/vins-covins/docs/run_COVINS-G.md#run_fe_vins

Best Manthan

Thank you very much for your response. I did not make any modifications to the code. I just ran the ORBSLAM3 stereo IMU node with the euroc open source dataset. As we know, COVINS only supports Mono IMU mode, so I wanted to try using the ORBSLAM3 stereo IMU mode with COVINS_G. Have you tried this before, or does the current code support it? Additionally, I noticed that the VINS+COVINS_G combination you provided is currently only available as a docker. Could you please advise on how to build it locally if I want to? My system is Ubuntu 18.04, so I shouldn't have any compiling issues.

[manthan99]

Hi

Unfortunately, I have not tried running ORBSLAM3 stereo IMU mode with COVINS_G, but I assume that if it doesn't work with COVINS, it will also not work with COVINS_G.

Regarding the VINS + COVINS_G, I can try guiding you but would be better if you can create a new issue for this and lets talk on that thread.

[NatureLan-sudo]

Hi

Unfortunately, I have not tried running ORBSLAM3 stereo IMU mode with COVINS_G, but I assume that if it doesn't work with COVINS, it will also not work with COVINS_G.

Regarding the VINS + COVINS_G, I can try guiding you but would be better if you can create a new issue for this and lets talk on that thread. Thank you very much for your reply. I have opened a new issue. https://github.com/VIS4ROB-lab/covins/issues/38

[ajxdhe]

Excuse me I met the issue when I build the Covins_g .......................................................................................................................................................................................................... Failed << covins_backend:make [ Exited with code 2 ]
Failed <<< covins_backend [ 54.9 seconds ]
[build] Summary: 17 of 18 packages succeeded.
[build] Ignored: 17 packages were skipped or are skiplisted.
[build] Warnings: 2 packages succeeded with warnings.
[build] Abandoned: None.
[build] Failed: 1 packages failed.
[build] Runtime: 1 minute and 6.0 seconds total.
So I ran cd /home/skywalker/ws/covins_ws/build/covins_backend; catkin build --get-env covins_backend | catkin env -si /usr/bin/make --jobserver-auth=3,4; cd - Then I met the issue that

[ 0%] Built target std_srvs_generate_messages_eus [ 0%] Built target tf2_msgs_generate_messages_nodejs [ 0%] Built target tf2_msgs_generate_messages_lisp [ 0%] Built target actionlib_msgs_generate_messages_py [ 0%] Built target actionlib_msgs_generate_messages_cpp [ 0%] Built target actionlib_generate_messages_py [ 0%] Built target actionlib_generate_messages_nodejs [ 0%] Built target actionlib_generate_messages_cpp [ 0%] Built target _covins_backend_generate_messages_check_deps_ServiceLoadMap [ 0%] Built target _covins_backend_generate_messages_check_deps_ServicePruneMap [ 0%] Built target _covins_backend_generate_messages_check_deps_ServiceGBA [ 0%] Built target _covins_backend_generate_messages_check_deps_ServiceSaveMap [ 0%] Built target roscpp_generate_messages_nodejs [ 0%] Built target tf_generate_messages_nodejs [ 0%] Built target tf_generate_messages_lisp [ 0%] Built target tf_generate_messages_cpp [ 0%] Built target roscpp_generate_messages_lisp [ 0%] Built target topic_tools_generate_messages_nodejs [ 0%] Built target topic_tools_generate_messages_lisp [ 0%] Built target roscpp_generate_messages_cpp [ 0%] Built target topic_tools_generate_messages_eus [ 0%] Built target topic_tools_generate_messages_cpp [ 0%] Built target actionlib_msgs_generate_messages_eus [ 0%] Built target dynamic_reconfigure_generate_messages_py [ 0%] Built target std_msgs_generate_messages_lisp [ 0%] Built target sensor_msgs_generate_messages_cpp [ 0%] Built target tf2_msgs_generate_messages_cpp [ 0%] Built target dynamic_reconfigure_generate_messages_lisp [ 0%] Built target std_srvs_generate_messages_lisp [ 0%] Built target dynamic_reconfigure_gencfg [ 0%] Built target pcl_msgs_generate_messages_cpp [ 0%] Built target nodelet_generate_messages_cpp [ 0%] Built target pcl_ros_gencfg [ 0%] Built target std_srvs_generate_messages_py [ 0%] Built target rosgraph_msgs_generate_messages_eus [ 0%] Built target std_msgs_generate_messages_py [ 0%] Built target geometry_msgs_generate_messages_eus [ 0%] Built target std_msgs_generate_messages_cpp [ 0%] Built target rosgraph_msgs_generate_messages_nodejs [ 0%] Built target tf_generate_messages_eus [ 0%] Built target rosgraph_msgs_generate_messages_lisp [ 0%] Built target std_msgs_generate_messages_nodejs [ 0%] Built target nodelet_generate_messages_lisp [ 0%] Built target actionlib_generate_messages_lisp [ 0%] Built target roscpp_generate_messages_py [ 0%] Built target std_srvs_generate_messages_cpp [ 0%] Built target rosgraph_msgs_generate_messages_py [ 0%] Built target topic_tools_generate_messages_py [ 0%] Built target roscpp_generate_messages_eus [ 0%] Built target dynamic_reconfigure_generate_messages_cpp [ 0%] Built target actionlib_msgs_generate_messages_nodejs [ 0%] Built target actionlib_msgs_generate_messages_lisp [ 0%] Built target tf_generate_messages_py [ 0%] Built target std_msgs_generate_messages_eus [ 0%] Built target actionlib_generate_messages_eus [ 0%] Built target rosgraph_msgs_generate_messages_cpp [ 0%] Built target geometry_msgs_generate_messages_cpp [ 0%] Built target geometry_msgs_generate_messages_lisp [ 0%] Built target dynamic_reconfigure_generate_messages_eus [ 0%] Built target geometry_msgs_generate_messages_nodejs [ 0%] Built target geometry_msgs_generate_messages_py [ 0%] Built target tf2_msgs_generate_messages_eus [ 0%] Built target sensor_msgs_generate_messages_py [ 9%] Built target covins_backend_generate_messages_eus [ 9%] Built target nodelet_generate_messages_eus [ 9%] Built target nodelet_generate_messages_nodejs [ 9%] Built target bond_generate_messages_nodejs [ 9%] Built target std_srvs_generate_messages_nodejs [ 9%] Built target nodelet_generate_messages_py [ 9%] Built target tf2_msgs_generate_messages_py [ 9%] Built target dynamic_reconfigure_generate_messages_nodejs [ 9%] Built target bond_generate_messages_cpp [ 9%] Built target bond_generate_messages_eus [ 9%] Built target sensor_msgs_generate_messages_nodejs [ 9%] Built target bond_generate_messages_lisp [ 9%] Built target nodelet_topic_tools_gencfg [ 9%] Built target sensor_msgs_generate_messages_eus [ 9%] Built target sensor_msgs_generate_messages_lisp [ 9%] Built target bond_generate_messages_py [ 9%] Built target pcl_msgs_generate_messages_eus [ 9%] Built target pcl_msgs_generate_messages_nodejs [ 9%] Built target pcl_msgs_generate_messages_lisp [ 9%] Built target pcl_msgs_generate_messages_py [ 17%] Built target covins_backend_generate_messages_nodejs [ 25%] Built target covins_backend_generate_messages_lisp [ 32%] Built target covins_backend_generate_messages_cpp [ 42%] Built target covins_backend_generate_messages_py [ 53%] Built target covins_base [ 53%] Built target covins_backend_generate_messages make[2]: No rule to make target '/home/skywalker/ws/covins_ws/src/covins_g/covins_backend/thirdparty/DBoW2/lib/libDBoW2.so', needed by '/home/skywalker/ws/covins_ws/devel/lib/libcovins_backend.so'. Stop. make[1]: [CMakeFiles/Makefile2:324: CMakeFiles/covins_backend.dir/all] Error 2 make: *** [Makefile:141: all] Error 2

[ajxdhe]

my system is Ubuntu 20.04

[manthan99]

Hi

Could you please create a new issue on the repo so it helps others having the same issue to find it by the title?

Best Manthan

[manthan99]

@patriksc Maybe you can already close this issue.

[ajxdhe]

@manthan99 OK I have already opened up the new issue