Onboarding Hello Tasklist

[anuragprat1k]

Type of Issue

Follow-ups from https://github.com/facebookresearch/droidlet/pull/475

• [x] Write an api for navigation (equivalent of this). Hello's translation python APIs are basic.
• [x] Test 'go to x' like commands
• [x] Get camera transforms
• [x] Test point cloud
• [ ] Write an obstacle detection heuristic to make exploration work
• [ ] Clean up duplication between the hello robot and locobot agents
• [ ] Rename Locobot semantics to more generic Robot semantics and have both Locobot and Hello Robot agents inherit from/use them.
• [ ] Expand Hello's capabilities - grasping, door opening etc.

Select the type of issue:

• [ ] Bug report (to report a bug)
• [ ] Feature request (to request an additional feature)
• [x] Tracker (I am just using this as a tracker)
• [ ] Refactor request
• [ ] Documentation Ask

[soumith]

also:

• flakiness in RGBDepth capture
• teleop is flaky
• (everything seems flaky)

So something wrt communication is flaky / blocking

[soumith]

• move_relative is not yet implemented

move commands need work because APIs for hello are not there yet. Hello move commands are based on translation / rotation

[soumith]

I tracked down part of the flakiness here: https://github.com/hello-robot/stretch_body/issues/44

[soumith]

posted a fix for it at https://github.com/hello-robot/stretch_body/pull/46

[soumith]

How does the status-quo for locobot work to move the base?

For locobot, the remote_locobot.py interface has a move_relative and move_absolute.

move_relative commands after entering the pyrobot side, are simply special-cases of move_absolute i.e., move_relative gets the current robot state in global co-ordinates, and then calculates the absolute world state of the relative position you are trying to get to — and finally it calls move_absolute with this new world state.

So, let's just focus on move_absolute. There are two types in which move_absolute is configurable under the hood by remote_locobot.

1. The Pyrobot-ROS pathway for everything: SLAM, planning, low-level control
2. droidlet's own DSLAM package for planning and SLAM, PyRobot-ROS for low-level control

1. Using PyRobot / ROS for high-level planning, SLAM, low-level control

In this case, move_absolute calls go_to_absolute in pyrobot (i.e. in base.py ).

This, in a very convoluted way via _execute_controller, simply calls _go_to_absolute.

Now, this function, finally uses the planner and base_controller to actually execute the moving to the given position.

Note: the planner would be used only if use_map is set to True here.

The planner does high-level path planning. It takes the target position, and a low-levelgoto closure that will let the planner call it with intermediate states to go to. By default, the MoveBase planner is used, which uses ROS to do planning. The planner needs some kind of ROS-based Visual SLAM configured. The default I think is ORB-SLAM2 or something.

The goto closure here is the base_controller's go_to_relative function, i.e. the actual low-level controller. By default, locobot uses the ILQR controller, which is implemented here.

A base controller (ILQR or otherwise) basically will take some fairly straight-line / obvious target position, make up a trajectory based on some math, and then go at it. It will keep reading the low-level servo/motor state and adjust it's trajectory. This is like advanced versions of the basic PID controllers that I'm familiar with. The plan for this low-level controller will be to give the linear and angular velocity(?) in xyz for each time step of the trajectory. In this base controller, ROS is used to publish this low-level control to the actual base / servos.

2. Using droidlet's own DSLAM package for high-level planning/SLAM

In this case, droidlet builds it's own map, obstacle detection and other stuff using the code [here]. droidlet will only use PyRobot for low-level control.

The planning is implemented by droidlet using the Scikit-FMM library. The planner is here.