search

b-it-bots / mas_domestic_robotics

Robot-independent ROS packages for domestic applications
GNU General Public License v3.0
28 stars 45 forks source link

Place action monitoring #219

Open alex-mitrevski opened 4 years ago

alex-mitrevski commented 4 years ago

Problem description

Our current place action doesn't have any monitoring and recovery behaviours, so the component always thinks that it has performed the action successfully.

We thus need to develop monitors for the action and, eventually, recovery behaviours as well.

Suggested solution

If an object is successfully placed, the scene will be stable, so the monitor I suggest would take consecutive images of the scene after placing and report successful placing if there is no significant variation between the scenes (I suppose optical flow can be used for this purpose). On the HSR, this can be done with the hand camera, but the monitor is not constrained to hand cameras (although for other cameras that have a larger field of view, background motion might affect the results significantly).

This monitor would not deal with all possible failures of the action (e.g. if a robot collides with the placing surface and releases the object on the floor), but it should be sufficient for the more common case in which the placed objects either falls from the surface or knocks other objects down.

@sthoduka Based on your experience with optical flow, can you foresee any particular challenges with such a monitor?

sthoduka commented 4 years ago

Is the arm going to be moving while it's monitoring? Some other things to keep in mind:

  1. sparse optical flow is cheaper than dense optical flow but needs nice features (corners etc.) to work
  2. image differencing is also a cheap alternative to OF