search

ros-navigation / navigation2

ROS 2 Navigation Framework and System
https://nav2.org/
Other
2.54k stars 1.28k forks source link

Characterize the navigation reliability - The Office Marathon 2 #929

Closed mhpanah closed 4 years ago

mhpanah commented 5 years ago

Objective Characterizing and investigating the reliability and performance of ROS 2 Navigation Stack

Description

The Office Maraton [1] was used on ROS 1 Navigation stack in an indoor office environment. For Navigation 2, we also need to perform similar kind of experiments to obtain better insights on the reliability, performance, and practicality of our nav stack in a real-world environment.

The following tasks are needed before conducting the Marathon Experiment:

[1] Marder-Eppstein, Eitan, et al. "The office marathon: Robust navigation in an indoor office environment." 2010 IEEE international conference on robotics and automation. IEEE, 2010.

crdelsey commented 5 years ago

Are there other metrics we'd prefer to use to quantify our reliability?

SteveMacenski commented 5 years ago

Checking that the local and global planners, costmaps are always making rate processing 3+ sensors would be good

SteveMacenski commented 5 years ago

Also, let me know if you have a time scheduled for this, I can marathon in with you remotely or find my way up to Portland

mhpanah commented 5 years ago

@SteveMacenski sounds great! We haven't scheduled anything for this task yet. We can talk about it more in our next WG meeting and from there schedule planning for this task.

SteveMacenski commented 4 years ago

Anyone have interest in writing a paper for this for IROS 2020? Its a little tight of a schedule but possible. I have lasers, cameras, and a few robot bases, but no substantially large space I can use. Maybe can coerce a co-working space to let me camp out for a week or another open-source friendly place

fmrico commented 4 years ago

Hi @SteveMacenski , I am interested in writing the paper for IROS2020, if you think it is fine, without obligation for your part. We can conduct the experiments in my lab, using kobuki or even Tiago. We got it navigating with navigation2 last week. We could extend the scenario even to the near corridors.

If you agree, you can email me to fmrico@gmail.com to coordinate.

crdelsey commented 4 years ago

I'm likely going to have limited time to work on this soon, so I don't want to commit to this. But this is a high priority for nav 2, IMHO. If there is anything I can do to support this - new features, bugs to look into, etc, let me know.

SteveMacenski commented 4 years ago

@fmrico sounds good! Lets chat about it at the weekly meeting tomorrow. We're moving it to 2pm because of the TSC meeting at 4pm PST. I'd like to be heavily involved in writing the paper. Given navigation2, along with the Intel folks, has been my baby since day 1. Ideally, I'd like to be the first author (one of the reasons I'm trying to drive this).

Given the deadline is relatively soon, and to do something similar to the Office Marathon, there's going to be a pretty substantial hardware burden (and tickets filed to make it work from inevitable crashes and failures) we should get started on this ASAP. The deadline is March 1 for regular paper submissions. That gives us about 6 weeks.

@crdelsey I'd love it if you and the other academically minded Intel folks were to be involved. It would feel weird to not have you guys as coauthors on the paper.

PDF of the Marathon in ROS1: http://ros.fei.edu.br/roswiki/attachments/Papers(2f)ICRA2010_Marder(2d)Eppstein/icra2010_marder-eppstein.pdf

I'd like to identify a few pieces of technology that we can use to aid in differentiators between the Marathon done back in 2010 and now. Particularly, I've ported STVL into ROS2 and that works with the stack, I think that's one example of a compelling change in algorithms. Obviously the behavior tree navigation is a big topic to cover as a differentiator. Maybe how ROS2 helps (memory use, determinism, reliability, ... more reliable). Fundamentally, not much else has really changed, though we make much greater use of depth cameras since 2010 than before. Less tilting hokuyos ;-). Lifecycle and determinism is a plus, but not sure its a huge topic to try to hit on. Personally, I would have liked to have replaced NavFn with something that computes paths based on kinematic knowledge of the robot, removing one of the most glaring failure conditions of ROS1 navigation, but I don't think that's in the cards in the next 6 weeks. DWB is still essentially DWA that's more configurable. Costmap2D is still essentially the same as well.

Hell, there's a non-zero chance I can convince Samsung to fly me to Madrid to help run and write this.

SteveMacenski commented 4 years ago

IROS_2020___Navigation2(9).pdf

Done, submitted for IROS 2020, up on Archix soon.

SteveMacenski commented 4 years ago

http://arxiv.org/abs/2003.00368

mhpanah commented 4 years ago

@SteveMacenski Hey Steve, It's great that this work got submitted to IROS. Let me know how the revision goes, I can help out with the BT and Recovery behavior sections.