search

osrf / rmf_core

Provides the centralized functions of RMF: scheduling, etc.
Apache License 2.0
102 stars 41 forks source link

How do I implement my own event in rmf? #271

Closed Marcusttk closed 3 years ago

Marcusttk commented 3 years ago

OS: Ubuntu 20.04

branch: master branch

I am looking to implement my own type of event in RMF, that can work together with rmf functions like the lift call, route planning, etc.

My event would be simulating computer vision capabilities on the AMR to influence its decision on whether it can safely enter the lift.

Hence I would need to implement an event in rmf that can pause the AMR and restart the AMR on its journey involving a lift, based on my computer vision output.

Please advise. Thank you!

mxgrey commented 3 years ago

Hi, @Marcusttk. It would be helpful if you could describe in greater detail the different kinds of situations you're anticipating and what the desired behavior would be, as well as what role the vision system will play in guiding that behavior.

Here are a few questions that may guide your elaborations:

  1. Is the vision system an integrated component of the AMR? If so, will its data be accessible through the API boundary of the AMR?
  2. What are the situations where we expect the robot will not be able to enter the lift? E.g. There are too many humans in the lift? There are other robots already in the lift?
  3. If the vision system determines that the robot cannot safely enter the lift, what should the overall behavior be? Should the robot sit and wait for the humans to make room for it? Should the robot cancel its lift request and try to summon the lift again later?
Marcusttk commented 3 years ago

Hi, @Marcusttk. It would be helpful if you could describe in greater detail the different kinds of situations you're anticipating and what the desired behavior would be, as well as what role the vision system will play in guiding that behavior.

Here are a few questions that may guide your elaborations:

  1. Is the vision system an integrated component of the AMR? If so, will its data be accessible through the API boundary of the AMR?
  2. What are the situations where we expect the robot will not be able to enter the lift? E.g. There are too many humans in the lift? There are other robots already in the lift?
  3. If the vision system determines that the robot cannot safely enter the lift, what should the overall behavior be? Should the robot sit and wait for the humans to make room for it? Should the robot cancel its lift request and try to summon the lift again later?
  1. Vision system will be accessible via the API of the AMR
  2. Too many humans in the lift. As a first version, we assume single AMR use case, before we move on to multiple AMR scenario.
  3. The robot will not enter the lift, and it will attempt to summon the lift again later.

I have some ideas on how to implement this according to the rmf framework. One idea would involve sending a "pause" signal to RMF. Is that doable?

So when the AMR reaches the lift holding area waypoint, and the lift has arrived, it would issue a "pause" command to RMF until the vision API has checked that the lift is good to go. The vision API would then output "go" or "no go" to the AMR, and the AMR would issue a "resume" signal to continue its RMF phases, depending on go/no-go

That is one way I can think of.

mxgrey commented 3 years ago

Thanks for elaborating on your expectations, that makes the problem much more clear.

I'm afraid to say that as it exists today, RMF is not suitable for having robots share lifts with humans in situations where there is a risk that humans will overpopulate the lift. There isn't a suitable code path at all for dealing with situation. In the current implementation, the handshake between the robot and the lift is meant to ensure that the robot will have exclusive rights to using the lift. "Try again later" is a simple concept for a human being to deal with, but it's an exceptionally challenging matter for robots which prefer to have very rigid and predictable sequences of events to follow.

We do have plans for robust lift sharing (e.g. have multiple robots in a lift simultaneously, and/or have robots and humans in a lift simultaneously) and a more intelligent lift scheduling system which this kind of camera system would work very nicely for, but that hasn't reached the top of our priority list yet. I expect to start working on it in 1-2 months from now if no other higher priorities pop up in the meantime.

So when the AMR reaches the lift holding area waypoint, and the lift has arrived, it would issue a "pause" command to RMF until the vision API has checked that the lift is good to go.

To address this idea directly, the trouble with this is the handshake between the lift and the fleet adapter will not allow the lift to leave until the robot has successfully entered the lift. We could pause the robot, but that means the people will have to exit immediately on that floor and give way to the robot or else everyone will be stuck forever.

As I mentioned before, we intend to replace this very rigid handshake with a more flexible lift request system, but we aren't there quite yet.

If you'd like to work on prototyping a solution before the smart lift request system is ready, I would probably suggest forking rmf_core and tweaking the rmf_fleet_adapter::agv::phases::RequestLift implementation. You could add a hook in that phase which listens for a "Go"/"No Go" signal, and if it gets the "No Go", then it could cancel its lift request and create a timer that reinitializes its lift request later. I wouldn't consider this to be a good long-term solution, but it could be a way to unblock and test out your system concept while waiting for the smarter lift system to arrive.

mxgrey commented 3 years ago

Closing as this is not an issue. To continue discussing this topic, please open a discussion.