How does “isPickedUp” work?

[jiachengc]

Hello,

I have an issue that the arm in manipulathor cannot output "isPickedUp" = True after I actually pickup an apple.

Now, I tried to manually test how the difficulty an arm can actually pickup a object in FloorPlan8 with hand radius = 0.1. I manually move the arm to the "Apple" by calling some navigation actions like, "RotationLeft", "MoveAhead", and extend the arm forward a little to nearby the target object Apple. Then, I tried to call action "PickupObject" to pickup the Apple, the Apple looks like be picked up by the arm, but it shows me False when I check object["isPickedUp"]. Also, I tried to print out the object["position"] and ,but it never change after I picked this Apple up.

Could you please tell me how the Mechanism behind this ["isPickedUp"]? how to determine the object is actually holding by agent's hand?

Also, Do you have any idea about my problem why this object["isPickedUp"] still false after I picked up the apple?

Looking forward for you reply, thanks a lot!

Before picked up, Apple["position"] = {'x': 0.9943692684173584, 'y': 0.9372280836105347, 'z': -1.1052862405776978}, Apple["objectId"] = Apple|+00.99|+00.94|-01.11

After picked up, Apple["position"] = {'x': 0.9943692684173584, 'y': 0.9372280836105347, 'z': -1.1052862405776978}, Apple["objectId"] = Apple|+00.99|+00.94|-01.11

@winthos

[winthos]

Hello!

I believe the issue is the isPickedUp bool returned in metadata is meant to be used with the default iTHOR agent initialization option. The iTHOR agent does not have an arm, and instead abstracts its pickup and drop actions in a way that teleports a targeted object to a predetermined position in front of the agent's camera. The isPickedUp flag is then set when this high-level action is performed with the default agent.

When using the ManipulaTHOR agent, we track and return picked up objects in a different way. Any successfully picked up objects are reported back in a list called heldObjects as denoted in the arm specific metadata here. The objects themselves are not flagged as picked up or not via the isPickedUp bool, as that is meant only to track objects manipulated by the high-level version of the pickup action.

Additionally, I believe you are not seeing a position change when picking up the apple because the pickup/drop actions with ManipulaTHOR only grab and drop the object, but do not reposition it automatically with a teleport like the high-level iTHOR version of the pickup action. Once the apple is successfully picked up, you should see it returned in the heldObjects metadata, and then you will only see a position change if you maneuver the arm, now holding the apple, to some different position and thus also change the position of the apple.

[jiachengc]

Thanks, very appreciate for you quick reply. very clear explanation.