search

askforalfred / alfred

ALFRED - A Benchmark for Interpreting Grounded Instructions for Everyday Tasks
MIT License
352 stars 77 forks source link

meaning of CoordinateObjectID #107

Closed nikita-petrashen closed 2 years ago

nikita-petrashen commented 2 years ago

Hi! How to get which instance of Potato do we interact with from these numbers? image

Side question: in tasks like pick_two_obj_and_place-PotatoSliced-None-GarbageCan-28 we need to slice two potatoes but in the original AI2-THOR scene we have only one potato instanced, what's the deal with that?

Thank you!

MohitShridhar commented 2 years ago

@nikita-petrashen,

How to get which instance of Potato do we interact with from these numbers?

Use objectID. This string uniquely describes each object in the scene.

Side question: in tasks like pick_two_obj_and_place-PotatoSliced-None-GarbageCan-28 we need to slice two potatoes but in the original AI2-THOR

In THOR (not the Unity Editor) interactable objects are spawned whenever a scene is instantiated. If the task involves two potatoes, then there must be two potatoes in the scene because the expert was able to complete the task during data generation.

MohitShridhar commented 2 years ago

ohhh sorry... @thomason-jesse just pointed out that for 'slicing' tasks maybe just one potato is sufficient because after a Slice action the potato is split into several slices, after which the agent can pick up two slices from the same pile (one by one).

nikita-petrashen commented 2 years ago

The thing is, THOR API is not suitable for my purpose, so what I'm trying to do is to manipulate the scene in the form of a point cloud using my own toolkit, thus creating a series of scene states which represents ALFRED's agent action sequence.

Back to potatoes, I have an item Potato_21b79aff in my scene located somewhere, and would like to know how to build a correspondence between these items in the scene and objectIDs in the trajectory descriptions.

Also, could you please hint the way how I can spawn several potatoes in the Unity itself? Maybe according to the scene part of the trajectory JSON? I cannot use THOR API for this because there is no way to get the point cloud representation of the scene from it, as was pointed in this issue.

Thanks!

thomason-jesse commented 2 years ago

That kind of AI2THOR manipulation is outside the scope of the ALFRED repository, but you might be able to get answers from the main AI2THOR repo. You could also check out the LUMINOUS framework for automatically generating new ALFRED-compatible scenes at the Unity level.