Create a find (object) action

[argenos]

The current action is embedded in the perceive plane action and should be refactored and separated to its own action server. This should also apply to other objects, for instance, shelves, and counters. It can potentially be even more generalizable, i.e. people.

[alex-mitrevski]

I agree that we need this action, but I don't think we can write it before having a decent knowledge base; without that, finding things will be a shot in the dark.

I'm talking about two types of knowledge here: encyclopedic knowledge (e.g. cups are stored in the kitchen), but also user-specific knowledge (e.g. Alex's cup is usually on the kitchen counter and not in the cupboard).

[argenos]

I agree first we need to at least fix #6, then solve this one. I guess for the world model we need to detect and recognize this objects first anyways, which was a bit what I was going for.

[minhnh]

So recently the object detection code is rewritten as an action server in this PR. Does that more or less solve this issue, or do you have some other thing in mind for the action?

[alex-mitrevski]

This is not an action about pure perception, but also requires a knowledge base (where things are likely to be stored so that we don't search blindly) and a recovery/exploration strategy in case the item is not found immediately.

PR https://github.com/b-it-bots/mas_domestic_robotics/pull/63 is an early-stage WIP for creating this knowledge base for the record.

[alex-mitrevski]

Now that the basic implementation of the ontology and the knowledge base interface is set up (https://github.com/b-it-bots/mas_knowledge_base), I finally have an initial draft of the action: https://github.com/b-it-bots/mas_domestic_robotics/tree/devel/mdr_planning/mdr_actions/mdr_perception_actions/mdr_find_object_action

The definition of the action allows finding either a specific object or an object of a given category, though for now, I'm only focused on specific objects.

The basic workflow of the action (in case of finding a specific object) is as follows:

• we first check if the object is already in the knowledge base (i.e. if we already know where it is); if yes, that is the result of the action (although we should actually verify that the object is still there)
• if the knowledge base doesn't have information about where the object is, we look for the default storing location in the ontology and we take that information as the result (although, just as above, we should verify that the object is actually there)

There's obviously still quite a lot to be done here, but it's a start.

[minhnh]

possibly related paper and the project it belongs to

• Kloss et al 2016, Learning where to search using visual attention
• Modeling Top-Down Saliency for Visual Object Search