search

open-rmf / rmf_ros2

Internal ROS infrastructure for RMF
Apache License 2.0
71 stars 57 forks source link

RMF Core Fleet Adapter Loop request bid proposal can't dispatch requests #141

Open chadlimjinjie opened 2 years ago

chadlimjinjie commented 2 years ago

Bug report

Required information:

Description of the bug

We have a working demo setup and another setup to integrate open rmf to our robots.

On the working setup when a Loop is send to rmf core, rmf core would create the loop task with BidProposal cost and then select that task as the winning bid which then dispatch loop request.

On our setup when Loop is send to rmf core, rmf core create the loop task with BidProposal cost and then doesn't dispatch the loop requests.

Questions

  1. When sending loop requests to rmf core does it require the time to select it as the winning bid?

Steps to reproduce the bug

Expected behavior

image Our setup should receive a dispatch request according to the winning bid.

Actual behavior

image We are not receiving a winning bid hence there is no /rmf_task/dispatch_request

Screenshots

image Bid Notice

image Bid Proposal

Additional information

mxgrey commented 2 years ago

The failure to receive a bid generally means that the task request was impossible for any of the fleets in your system to perform.

For a loop task this usually means that you've asked the robot to loop between two waypoints that aren't connected by any path on the navigation graph. Usually when two points on the navigation graph aren't connected, it's because of a small, accidental break in the navigation graph. See this post for an example of what I mean: https://github.com/open-rmf/rmf/discussions/59#discussioncomment-866410

I recommend closely double-checking your navigation graph in the traffic editor and seeing if there are any lanes whose waypoints are accidentally disconnected.

chadlimjinjie commented 2 years ago

Hi @mxgrey

From the setup we have our fleet adapter is already sending the BidProposal with a valid cost value, this means the 2 way points are correctly connected. Nonetheless, the demo setup is working using the same path on the same building.yaml file.

We still don't quite understand the criteria to determine the winning bidder as well as would the submission time matter for the selection of the winning bid.

Yadunund commented 2 years ago

I noticed that the requested start time for your loop request with real robots (without gazebo) is still description: {start_time: {sec:0 , nanosec:0}}. This actually corresponds to the start of unix time (January 1, 1970). So you should be populating this time with the present unix time. But this should not have an impact in the dispatcher publishing a DispatchRequest. There was an issue before related to supporting start_times well in the past but this has been fixed. Could you check if your "internal copy of RMF" as stated in your description includes this fix? Are you using binaries or building from source? Your description states both which is confusing.

More importantly, could you share the entire output of the BidProposal message? Specifically highlighting these fields? Also for a sanity check, is the use_sim_time parameter set to false for all RMF and fleet adapter nodes that you are running? Also could you try the latest main for all the RMF repos and let us know if your problem still persists.

mxgrey commented 2 years ago

would the submission time matter for the selection of the winning bid.

I think I understand the issue now. Unfortunately right now there's a hardcoded cut-off of 2 seconds for all bids to arrive. We've generally found that 2 seconds is usually an order of magnitude more time than a bid usually takes to arrive, so we're using this hardcoded timeout until we have a chance to change the implementation to a more sound handshake-based system.

If your task bid takes an unusually long time to generate, you could try extending the 2 second timeout. Or a better stop-gap measure would be to add a ROS parameter for that timeout. This problem will be fixed correctly when we improve the bidding system, but forcing the bid window to be bigger should help in the short term.

mxgrey commented 2 years ago

It was just pointed out to me that the bidding time is already a ROS parameter. You can set it by modifying this field in the common.launch.xml.

chadlimjinjie commented 2 years ago

I noticed that the requested start time for your loop request with real robots (without gazebo) is still description: {start_time: {sec:0 , nanosec:0}}. This actually corresponds to the start of unix time (January 1, 1970). So you should be populating this time with the present unix time. But this should not have an impact in the dispatcher publishing a DispatchRequest. There was an issue before related to supporting start_times well in the past but this has been fixed. Could you check if your "internal copy of RMF" as stated in your description includes this fix?

I'll have to check with my team on what version we are using. But currently I am using the 'feature/integration-test' branch the winning bid is still not selected and no DispatchRequest is published.

Are you using binaries or building from source? Your description states both which is confusing.

I'm currently using https://github.com/open-rmf/rmf 'feature/integration-test' branch build from source.

More importantly, could you share the entire output of the BidProposal message? Specifically highlighting these fields? Also for a sanity check, is the use_sim_time parameter set to false for all RMF and fleet adapter nodes that you are running? Also could you try the latest main for all the RMF repos and let us know if your problem still persists.

Here are the logs from the BidProposal message.

image without Gazebo

image without Gazebo use_sim_time parameter set to false.

cnboonhan commented 2 years ago

hello,

is there a specific reason for choosing to use integration-tests branch? I made the branch as a legacy "bookmark" to allow some previous tooling to work, but I would strongly recommend working off something more recent if possible.

Would also recommend uninstalling any / all rmf binaries if you are also building RMF from source to prevent overlay confusion:

sudo apt remove ros-$ROS_DISTRO-rmf-*

should do the trick.

KaeYan commented 2 years ago

So you should be populating this time with the present unix time. But this should not have an impact in the dispatcher publishing a DispatchRequest.

@Yadunund if i set the use_sim_time to false, how can we config the fleet adapter to use unix time?

Also, when we send fleet states, we want to use unix time. Say if the unix time now in nanoseconds is 1637302549019356595. How should we update the Time in Location? Is it secs = 1637302549, then nanoseconds = 1637302549019356595?