Activity Interface - reachability with a tool is not kept when tool is grasped

AlexAntn commented 9 years ago

This problem is similar to the problem we faced on #151

@vtikha Could you please do the same thing for reachableWith? If you need any examples/clarifications, let us know and we will obtain them

vtikha commented 9 years ago

@AlexAntn what do you mean "similar problem as #151" and with "do the same thing"? this is a totally different problem. I cannot understand what the problem is exactly, give examples of your issue.

gsaponaro commented 9 years ago

I think that @AlexAntn meant the following:

If an object O is reachableWith A (i.e. the list returned when doing reachableWith O contains A) and then the robot grasps O, after this operation the list returned by reachableWith O must still contain the element A. In other words A must be persistent in time if it was true "before the grasping action".

@AlexAntn is it like this?

AlexAntn commented 9 years ago

@vtikha when a tool is taken by the iCub, the list of objects reachable with that tool is cleared.

Example: we have "cheese" reachable with "rake". We then instruct the iCub to askForTool rake, and after it takes the tool, the "cheese" no longer appears as "reachableWith rake".

vtikha commented 9 years ago

@AlexAntn I am trying to understand the issue here, but it is still slightly unclear. Anyway I am currently implementing a more robust reply, by checking what is in the scene and in the hand etc...But I have a few questions:

if you have all the objects in front of the robot and query reacheableWith cheese while the cheese is far the reply could be (if only the rake is available) rake left or rake right or rake left right then take the tool.
- Querying again might be redundant but this might be useful if while the robot is taking the tool somebody changes the scene. In any case, the new version I am implementing can deal with this situation even if the tool is not present in the scene (holding).
How do you expect the reply to be if the object is close?
How do you expect the reply to be if the object is far and no tools are available.
How do you expect the reply to be if you query an object that is not available?

Any other info I should be aware of before committing the changes?

AlexAntn commented 9 years ago

@vtikha Ok, replying point by point:

1) When i query reachableWith cheese while the cheese is far, the reply should be a bottle with the objects the iCub can use to reach the object. If only the rake is available, then the reply should be only "rake". We have not considered the reachability depending on which hand the tool is. That reply ("rake right" or "rake left") would lead the iCub to think it could reach with both rake and right/left hand. The problem we encountered had to do with querying while the tool was already in the hand, in which case it would consider the tool for reachability. If the new version fixes this, then it's no longer a problem!

2) If the object is close, the reply would be "right left rake", if only the rake is available.

3) If the object is far and no tool is available, then an empty bottle should be returned

4) If a query is made for an object that is not available/doesn't exist, a [fail] should be returned.

I think this is it. @gsaponaro Are these the replies you expect on the world state manager?

vtikha commented 9 years ago

@AlexAntn The issue here and questions I asked are about what you actually expect form the reachableWith, not problems as this is a totally different function now. Bare with me but I have issues with implementing something that does not makes sense to me.

1) & 2) I mentioned the left and right as this is how it was decided to be, and, logically, reachability should be considered depending on the hand or even which hand the tool is . So... if you have multiple objects on the table,( consider all within reach & nothing in the hand)...bun-bottom bun-top cheese etc etc you want the reply to be a list containing all the objects? How is this even useful? Then what would you expect the robot to do? pick up the meat in order to reach and take the cheese? The logical thing that comes to mind is that if the cheese is within reach for the robot, then the reply would be left and right (meaning left hand or right hand), we can even discriminate if the object is closer to the left or the right hand and remove one if needed. The whole list makes no sense to me (considering the robot) but if this is what you like it is easy to add. Then the reply would be (left right obj1 obj2 obj3 ...) If on the contrary the cheese is far (unreachable by hand) then the scene is analysed and only a tool like object (elongated or whatever) would be returned. 3) if nothing useful is found then the reply is empty...agreed. 4) agreed.

Lastly, if the tool, consider the rake, is in the left hand and the cheese is still far (hence why querying again), then the reply should only have rake or even left rake (the hand that holds the tool and the obj). Correct?

gsaponaro commented 9 years ago

I think this is it. @gsaponaro Are these the replies you expect on the world state manager?

Short answer: yes. In this issue we are discussing the semantics of that query, the desired behaviour in a number of situations. I elaborated a bit on the syntactic aspects of how those responses are treated by the rest of the pipeline (worldStateManager -> opc2prada -> PRADA) in a separate issue, #165.

AlexAntn commented 9 years ago

@vtikha 1) & 2) On the previous version of this function, we did not consider in which hand the tool is when checking for reachability, hence my confusion. All the pipeline is currently working on the assumption that it doesn't depend on which hand has the tool, as you can see on issue #165. As for the list returned, we can omit anything that is not a tool or a hand, as it won't be used in any example. The idea was to provide all the information available to the robot/planner, and it would decide, but i suppose we can simplify.

In short: If an object is close, the reply would be "right left". If an object is far, the reply would be "rake".

As for the last question: The reply should be "rake"

I suggest we make a skype meeting to discuss if we should consider which hand the tool is in when querying for reachability, as it involves changing more than just one module. When will you be available, @vtikha ?

gsaponaro commented 9 years ago

Hi @vtikha, any comments in response to the latest message by @AlexAntn about this issue?

gsaponaro commented 8 years ago

Hi @vtikha, I know some changes with reachable/pullable are in progress, anyway, we spotted a couple of problems: (i) duplicates, (ii) empty strings. Example:

screenshot from 2016-02-16 11 27 36

>>reachableWith cheese
Response: (rake rake bun-bottom "" left right)

vtikha commented 8 years ago

@gsaponaro, as you know i am still working on it. I am aware of errors, no need to mention them. There are many others that need to be fixed from your side. Will let you know when I will have time to finish it, and it is ready.

gsaponaro commented 8 years ago

I was not sure whether you knew about both of those problems. OK then, talk to you later!

vtikha commented 8 years ago

@gsaponaro Should now behave as expected. Let me know

AlexAntn commented 8 years ago

Hello @vtikha , we have ran some more tests, and we found the following issue:

When an object is far away from the robot (so, not reachable with its hands), the reply to reachableWith is a bit unstable.

We have taken a screenshot to illustrate the issue: 160219reachable1

(as shown above, sometimes it would reply rake, as it should, other times it would reply left right)

The segmentation and labelling of the objects were both stable.

vtikha commented 8 years ago

@AlexAntn please show me the output of activityInterface (as you seem to be running everything through terminal) I am trying to contact you in skype. It seems that the bun-top is within the limit of out of reach. Let me know when you will be available.

AlexAntn commented 8 years ago

@vtikha , here go the outputs:

reachableWith meat: reachable_meat

reachableWith bun-top: reachable_bun-top

And also, the output of actvityInterface: actIntOutput.txt

vtikha commented 8 years ago

There seems to be some issues with OPC...sometimes it returns a position: 0.000000 0.000000 0.000000. The positions seems to jump a lot, when you did the trial did you move the objects or were there fixed on the table? Plus I see a lot of packets being dropped: please connect everything to activityInterface as tcp.

AlexAntn commented 8 years ago

Hello @vtikha ,

I did not move the objects at all during this last trial, i just sent rpc requests using the activityInterface rpc.

I re-run the trial, after changing all connections to TCP. It no longer has any package losses, but the reachableWith is still unstable.

Looking at the output, i can still see the OPC returning a position: 0.000000 0.000000 0.000000.

I will restart the whole thing, and see if the problem persists.

vtikha commented 8 years ago

I do not know what is going on there, as I never had this while testing here. You are getting weird of positions even 20 cm...this is crazy...the objects are stable and this is not possible....

AlexAntn commented 8 years ago

Hello @vtikha ,

I restarted everything, rebooted all the machines involved, and re-ran the modules.

I ran the trial again (querying reachableWith on meat and bun-top), and the result is slightly more unstable than previously.

Any suggestions on any tests we can perform to debug this?

vtikha commented 8 years ago

slightly more unstable? what does this mean? This was tested here in various conditions with various tools and it was behaving correctly. I have no idea how to debug this. Your system is super slow right? Maybe it is a bad sync of all the modules. Try removing objects, start with only a tool and a object, then add one at a time and try to see where things go wrong. I cannot try it here as we just packed the robot to ship to Karlsruhe for a review meeting.

AlexAntn commented 8 years ago

Well, all the queries i make, even with only two objects, return something like this:

[INFO]position is -0.439939 -0.024914 0.144159 [ERROR]after all: rake [ERROR]will now send: rake left right

or

[INFO]position is 0.000000 0.000000 0.000000 [ERROR]after all: rake [ERROR]will now send: rake left right

(this was for only rake an bun-top on the table)

This looks like it is indeed probably a very slow network issue. I see no other reason for this to get worse after the reboot.

AlexAntn commented 8 years ago

Hello again, @vtikha ,

We ran some more trials, this time dumping the information of the OPC, and the positions are coherent (except when they are 0.00000, of course, but this might be due to flickering and/or bad connection).

Is there a possibility that this is due to calibration issues? Since both 2D and 3D positions are going around in the pipeline, a miscalibration of the cameras might result in the two different coordinates not pointing to the same object.

vtikha commented 8 years ago

@AlexAntn @gsaponaro The last commit should have fixed the 0.0 issue.

vtikha commented 8 years ago

I am closing this for now, as we confirmed it is solved

robotology / poeticon

Activity Interface - reachability with a tool is not kept when tool is grasped #163