search

devendrachaplot / DeepRL-Grounding

Train an RL agent to execute natural language instructions in a 3D Environment (PyTorch)
https://sites.google.com/view/gated-attention/home
MIT License
237 stars 37 forks source link

Weird Targets for train instructions #9

Closed y12uc231 closed 6 years ago

y12uc231 commented 6 years ago

We are looking into some train instructions and there are few things which are strange. Would be great if someone clarifies this. All the instructions mentioned below are from instruction_train.json.

{ "instruction": "Go to the red object", "targets": [ "ShortRedTorch", "ShortRedColumn", "TallRedColumn", "RedTorch", "RedSkull", "RedCard", "BlueArmor" ], "description": "red object" } How come "BlueArmor" be a red object?

{ "instruction": "Go to the smallest green object", "targets": [ "GreenArmor", "TallGreenColumn", "ShortGreenTorch", "ShortGreenColumn", "GreenTorch" ], "description": "smallest green object" }

How come "TallGreenColumn" be a small object?

There are several such cases in the instruction set.

dheerajrajagopal commented 6 years ago

  1. BlueArmor - VizDoom issue. When you actually render this in Vizdoom, it gives a red armor not blue. We had to put in an exception

  2. TallGreenColumn - this is to mainly handle the cases where the only green object in the env is a "TallGreenColumn" and the right answer to "smallest green object" would be this. If there are multiple green objects selected, then we identify the smaller sized green object and use it in the environment.

Hope that clarifies.

y12uc231 commented 6 years ago

I see. Thanks for clarifying.