Visual SLAM (or non-Lidar positioning) demo

[SteveMacenski]

Using the T265, SLAMCore, or another flavor of visual / depth / inertial / ... SLAM that doesn't rely on a lidar for use in the navigation stack.

The goal being to clearly dispel the incorrect notions that navigation only works with lidar based systems. While lidar based systems are more robust, it is possible to do do it as long as you find some visual slam vendor that meets your needs, irregardless of navigation2.

This may need to include creating something to use depth images to make a 3D reconstruction of the space using the posts from V-SLAM and readings from depth / stereo so that we can use that 3D map to navigate with the global costmap. Essentially we should reconstruct -> save to file -> load from file in next session for localization -> input into costmap as a pointcloud / new 3D static layer so the planner can plan through some representation of the space. This could also be done with a dense feature map if available or other outputs of a particular V-SLAM application.

[SteveMacenski]

https://ramyadhadidi.github.io/files/shoghi_src_esweek.pdf

Plan for now is to start with ORB-SLAM2 because we know it works and we have a maintainer on it wiling to help with a ROS2 port and support. Then we can explore other options like Kimera and OpenVSLAM that we don't necessarily know will work or don't have a localization mode.

The paper above also shows with minimum configuration it can work on a raspberry pi sized embedded computer

[siddhya]

I would like to contribute towards this task. But I do not have the T265 or other hardware.

[SteveMacenski]

Lets touch base on Slack. We were talking about it on the mapping and localization channel but that's moved onto the new localization framework topic. The current state of things if we have the basic ROS2 port for ORB-SLAM2 done, but we need to expose the occupancy grid / feature map from the ORB-SLAM2 to the ROS wrapper to publish as pointclouds / other messages we can use in navigation for collision avoidance. I believe that is the final step for getting ORB-SLAM2 ready.

Then to use that information, we need to do 2 things (discussion on the perception channel currently):

1. Update the map server to have a 3D map IO library (using pcl::io module for pcd files) to load the maps from file when saved for localization-based navigation on a pre-generated map (also useful for 3D SLAM)
2. a new static 3d costmap layer to take in that pointcloud map topic and place into the occupancy grid for navigation.

Then we should have all the tech we need to do some demos with ORB-SLAM2 specifically! But then also all the capabilities needed in the nav2 stack for any V-SLAM or 3D SLAM. We could even look at other V-SLAMs to do tests with in this way.

[KUTuaNithid]

Hello. I'm interested in your work. Now, I'm trying to integrate openvslam into ros navigation stack. Could you tell me more about your work. It may help me a lot. Thank you

[SteveMacenski]

Many of our discussions also take place on our community slack https://join.slack.com/t/navigation2/shared_invite/zt-hu52lnnq-cKYjuhTY~sEMbZXL8p9tOw, feel free to join and introduce yourself on on-boarding channel.

On this topic, we haven't made much progress in recent months. We've been refocused on other topics, but if this interests you, I'd be more than happy to chat and figure out how we can work through this for the benefit of all Nav2 users. We were thinking about standardizing around ORBSLAM2 or ORBSLAM3 since they're very stable and shown to work on larger mobile robotics scales. Does openVSLAM have a similar level of stability / functionality?

I glanced over the paper and it does look interesting, though they compare against ORBSLAM(1) not 2 which is more commonly used or now the new 3 which has additional improvements. I agree with their sentiment in their paper about issues with open source visual slam libraries, though performance on industrial mobile-robot scale VSLAM problems is first and foremost the most important thing. I'm not saying that openVSLAM wouldn't work, but it requires some evaluation (looks promising).

Would definitely be interested in chatting more about this and see if you'd be interested in contributing by helping integrate vslam into Nav2.

Ref: open vslam https://arxiv.org/pdf/1910.01122.pdf orb slam 3 https://arxiv.org/pdf/2007.11898.pdf

[SteveMacenski]

We've done some detailed analysis on ORB-SLAM3, OpenVSLAM, and RTABMap and have decided on OpenVSLAM for integration into Nav2 for the following reasons (summarized from a larger analysis)

• OpenVSLAM operates the best under the most broad set of conditions (outdoor urban, indoor warehouse, low-lighting) over all of the other methods
• It supports Mono, stereo, RGBD, fisheye, and equirectangular lens
• It has permissive licensing allowing us to work on it and for it to be integrated into a commercial project / research (BSD2)
• There's an active community around it and will be able to merge PRs and make improvements to it to aid in navigation development (CC @ymd-stella)
• While it lacks IMU fusion techniques, we found that those did not represent significant improvements in our analysis. ORB-SLAM3 does support this, but it fails terribly in degraded lighting conditions where openVSLAM works better. This would be great to have added or work on adding to openVSLAM though.
• While rtabmap provides occupancy grid information, it has by far the lowest accuracy of all the techniques analyzed and is only on par with 2D low-resolution SLAM (at best). The other techniques analyzed well beat this with roughly 5cm accuracy

[soldierofhell]

https://github.com/xdspacelab/openvslam/wiki/Termination-of-the-release ?

[SteveMacenski]

We’re using the community fork where this topic has been discussed at length. Please review that, I’m not going to mix discussions on different projects in here

[m2-farzan]

I want to give it a try. Can I get an update before I start? Is anyone working on this?

Also, couple of questions (@SteveMacenski):

1. I read your article on comparing V-SLAM tools (nice article by the way), and it says that OpenVSLAM is way to go with Nav2. Now we have OpenVSLAM-community fork but from slack I found that LP-SLAM is being considered too. Has a final decision been made yet?
2. Should the demo be in form of A) simply a tutorial in navigation.ros.org or B) tutorial + nav2_bringup extension (e.g. launcher param vslam:=True) or C) a standalone package, e.g. nav2_vslam, or something else?
3. Neither LP-SLAM nor OpenVSLAM-community seem to have binary releases. So the demo is going to include build steps, which is in contrast to other navigation.ros.org tutorials. Is this fine for now? Is there anything we can do about it?

[SteveMacenski]

@AlexeyMerzlyakov would be in the best place to give you a full explanation of this work.

LP-SLAM is based on OpenVSLAM + adding some more features to it that are supposed to help with the mobile robot use-cases. But we haven't totally abandoned OpenVSLAM. There's not a clear explanation to offer right now, this is still on-going work that could go either way.

The output of this work would be such a tutorial, so we haven't gotten there yet. But if you wished to help in writing a tutorial about swapping out and using another localization method, I think we'd be happy to have it! Actually another ticket #2728 is basically asking for the same thing (e.g. how to replace AMCL / SLAM Toolbox with my preferred localization / mapping system).

Yes, it involves building from source.

[ViniciusAbrao]

Hi @SteveMacenski ,

any news about this topic?

[m2-farzan]

My progress report FWIW:

I've been writing a draft version of the tutorial (wip in my fork). My current problem is, the map representation used by OpenVSLAM is in the form of key-frames and landmarks, which is different from the occupancy grid format used by Nav2. Looks like we need to implement a feature in OpenVSLAM-ROS that converts their map format to occupancy grid. It's not that trivial. I've opened a discussion for this, hoping to get tips before starting to work on the feature.

Also in the meanwhile I'm trying to create deb packages for openvslam to achieve a reproducible tutorial (also to make life easier for openvslam users). The work can be found here, but I'll probably move it to ubuntu ppa.

[ViniciusAbrao]

So, the plan is to keep working with OpenVSLAM even in the case of possible license discussions ? I do not want to bring the discussion to here, only understand what are the plans and next steps.

[SteveMacenski]

So for clarity, we (e.g. @AlexeyMerzlyakov + a few other volunteers in the Nav2 working group) spent a great deal of time to get some VSLAM systems working for a production-ready system with VSLAM. We had some prototypes of occupancy grid generation systems using the feature map and key frames to back out a 2D grid for use in navigation.

However, after our case study on VSLAM techniques, we found that the openly available / academics are still not yet ready for much more than short-term POCs or single-session localization use so we have stopped working on this project and reallocated resources to more realistic goals where the technology either (1) already exists or (2) we have the resources to create it ourselves.

As such, I am closing out this ticket since the roadmap forward on this would be on the scale of a 10 year project when more VSLAM technologies are around and ready for practical use to have integration with Nav2 as a localization system option.

[mkolodziejczyk-piap]

Sorry for complaining, but IMO the effort was wrongly directed to SLAM system that beside "Open" term in name (which has good associations) is rather mediocre and not implemented with ROS extension in mind, not to mention navigation application (and as it turned out, the whole "implementation" was questionable). IMO better was to take rtabmap_ros as a starting point, fix integration with nav2 and optionally improve its SLAM capabilities (e.g the "inertial" ones). rtabmap_ros was successfully used with ROS1 nav stack and has many handy tools already implemented (vide http://wiki.ros.org/rtabmap_ros/Tutorials/StereoOutdoorNavigation)

[Hermanoid]

For reference, the "community fork" above appears to refer to stella_vslam: https://github.com/stella-cv/stella_vslam This is a redirect from the link in Macenski's paper: https://github.com/OpenVSLAM-Community/openvslam