allenai / ScienceWorld

ScienceWorld is a text-based virtual environment centered around accomplishing tasks from the standardized elementary science curriculum.
https://sciworld.apps.allenai.org/
Apache License 2.0
223 stars 26 forks source link

More info needed for solving ambiguous requests #11

Open laetitia-teo opened 2 years ago

laetitia-teo commented 2 years ago

Hi !

I'm using ScienceWorld in interactive mode, and I want to make some violet paint, by mixing blue and red paint. I go to the art studio, and I try to pour red paint in wood cup.

The contents of the art studio are:

This room is called the art studio. In it, you see:

    a large cupboard. The large cupboard door is closed.
    a table. On the table is: a jug (containing nothing).
    a wood cup (containing red paint)
    a substance called air
    the agent
    a wood cup (containing blue paint)
    a wood cup (containing yellow paint)

You also see:

    A door to the hallway (that is open)

And this gives me the following observation:

Ambiguous request: Please enter the number for the action you intended (or blank to cancel): 0: pour red paint (in wood cup, in art studio) in wood cup (in art studio) 1: pour red paint (in wood cup, in art studio) in wood cup (in art studio) 2: pour red paint (in wood cup, in art studio) in wood cup (in art studio)

The last observation is not informative enough to take specify one action to take to pour the red paint in the container containing blue paint.

PeterAJansen commented 2 years ago

Thanks for this note. I struggled a bit with this -- creating the ambiguity resolution message -- in trying to balance a generic method that would give some distinguishing feature, but not be too verbose. In the end, settling on the direct container that an object is in seems to work 80% of the time, but sometimes (as you note) it's still not ideal. Very welcome to hear suggestions and/or alternatives.

The gold path generation does include a method for generating unique referents -- I suppose it could be possible to take the set of referents for each ambiguous object, find the ones that are different between them, and then use those -- though the list of allowed referents can be quite large (and some can be a little unusual, since they're generated automatically using common patterns), so this might occasionally generate some unusual names.