Perception pipeline

[jeking04]

Now that we have both apriltags and rock I think we are ready to implement a perception pipeline. This issue is for discussing concrete implementation details related to this.

The current idea is to follow the planning pipeline. Here is a stab at some of the implementation details that we can do right now:

1. Create a prpy/perception python module
2. Implement a PerceptionMethod that parallels PlanningMethod
3. Create a prpy/perception/rock.py that implements DetectObject - this will be a PerceptionMethod
4. Create a prpy/perception/apriltags.py that implements DetectObjects - this will also be a PerceptionMethod
5. Update the robot to have something like robot.detector which parallels robot.planner and gives an ordering that the perception modules should be tried
6. Add the ability to call robot.

Future:

1. Create a simulated perception module. If the robot is started in sim mod, set robot.detector to point to this simulated perception module.
2. Add DetectObjects to rock

Okay, I think this captures most of what we discussed during the rock demo today. What do people think?

[psigen]

Wow, this sounds like a great idea. I like paralleling PlanningMethod, perhaps we can make a PrpyMethod base implementation for adding these reflexive method chains.

[siddhss5]

Sounds great! A few points:

• Ideally, prpy/perception should handle all robot-specific things, including extrinsics and mapping perception objects to kinbodies.
• In my mind, there's some YAML or other file called rock.yaml that contains the dict that maps kinbodies to perception objects. The base class then contains an object dict that loads that.
• I also envisioned the DetectObjects function to take an optional list of kinbodies, if we want to focus the search for specific object sets.
• In the future, we can be fancy about this: given a requested object to detect, we can automatically figure out which of our perception algorithms support its detection and call them specifically [or in some order].

[jeking04]

Okey dokey,

I have started these changes in this branch: https://github.com/personalrobotics/prpy/tree/feature/perception_pipeline

and the associated herbpy branch of the same name.

This currently just calls the kinbody_detector, which is a separate package, and lets the kinbody_detector load objects into the environment. Next we can work on restructuring that package and pulling some functionality into prpy (per Bullet 1 above). But have a look at that branch and make sure structurally its what we have in mind.

Next up I am going to try to create a rock detector. Then we will need something that parallels the Sequence or Ranked MetaPlanners. I think this goes along the lines of bullet 4.

[siddhss5]

@jeking04 and I just had a chat about where to place the logic and code that updates the OpenRAVE environment. Note that this logic can be fairly complicated and possibly custom for different demos/objects and would have to handle things like:

1. Persistence of objects that are not in the FOV.
2. Resolving IDs of multiple instances of the same object.
3. Filtering over time and across multiple detectors.

My suggestion is to create a perception/worldupdater.py [or something better named] that holds the logic for updating the world that takes in an env and possibly a list of whatever the detector[s] output[s] and updates the env.

[athackst]

anyone else noticing prpy.perception isn't showing up as a module in python? I had to add some references to it in setup.py and init to see it

[mkoval]

Added in #189 as part of the block sorting demo.