microsoft / TextWorld

​TextWorld is a sandbox learning environment for the training and evaluation of reinforcement learning (RL) agents on text-based games.
Other
1.23k stars 188 forks source link

Can we generate full description at each step? #298

Closed junkyul closed 2 years ago

junkyul commented 2 years ago

Hello, is there a way to generate a full description of each step instead of observing the changes since the previous step?

For example, we see an observation at the beginning of a game like

-= Kitchen =- You've seen better kitchens, but at least this one seems pretty standard.

You make out an opened fridge in the corner. The fridge is empty, what a horrible day! You can see a kitchen cupboard. There's something strange about this thing being here, but you don't have time to worry about that now. The kitchen cupboard contains a dirty pie plate. You smell an interesting smell, and follow it to a trash can. The trash can is empty, what a horrible day! You can see an opened cutlery drawer. The cutlery drawer is empty! This is the worst thing that could possibly happen, ever! You can see an opened dishwasher. What a letdown! The dishwasher is empty! You bend down to tie your shoe. When you stand up, you notice an oven. The oven is empty, what a horrible day! Oh wow! Is that what I think it is? It is! It's a dining table. You shudder, but continue examining the dining table. The dining table is massive. Unfortunately, there isn't a thing on it. You can see a ladderback chair. The ladderback chair is solid. Unfortunately, there isn't a thing on it. You can make out a stove. But the thing is empty, unfortunately. You move on, clearly furious with your TextWorld experience. You see a counter. The counter is vast. Looks like someone's already been here and taken everything off it, though. You can see a dining chair. Now why would someone leave that there? The dining chair is solid. Unfortunately, there isn't a thing on it. Aw, here you were, all excited for there to be things on it!

Then, after some action we observe

You take the dirty pie plate from the kitchen cupboard.

Would it be possible to get the full-textual description of each state from the internal game state?

-= Kitchen =- You've seen better kitchens, but at least this one seems pretty standard.

You make out an opened fridge in the corner. The fridge is empty, what a horrible day! You can see a kitchen cupboard. There's something strange about this thing being here, but you don't have time to worry about that now. “You take the dirty pie plate from the kitchen cupboard.” You smell an interesting smell, and follow it to a trash can. The trash can is empty, what a horrible day! You can see an opened cutlery drawer. The cutlery drawer is empty! This is the worst thing that could possibly happen, ever! You can see an opened dishwasher. What a letdown! The dishwasher is empty! You bend down to tie your shoe. When you stand up, you notice an oven. The oven is empty, what a horrible day! Oh wow! Is that what I think it is? It is! It's a dining table. You shudder, but continue examining the dining table. The dining table is massive. Unfortunately, there isn't a thing on it. You can see a ladderback chair. The ladderback chair is solid. Unfortunately, there isn't a thing on it. You can make out a stove. But the thing is empty, unfortunately. You move on, clearly furious with your TextWorld experience. You see a counter. The counter is vast. Looks like someone's already been here and taken everything off it, though. You can see a dining chair. Now why would someone leave that there? The dining chair is solid. Unfortunately, there isn't a thing on it. Aw, here you were, all excited for there to be things on it!

Thanks!

MarcCote commented 2 years ago

TextWorld doesn't provide text that includes the game feedback integrated into the room observation. However, what people have been doing is concatenating (at each game step) the game's feedback, the room's description, and the player's inventory.

You can achieve that by requesting additional information from the environment (see textworld.EnvInfos) like so:

import textworld
requested_infos = textworld.EnvInfos(feedback=True, description=True, inventory=True)
env = textworld.start("path/to/game.z8", requested_infos)
state = env.reset()

# Build the observation string. 
observation = state.feedback + "\n" + state.description + "\n" + state.inventory
print(observation)

# Take an action.
state, score, done = env.step("drop hat")
observation = state.feedback + "\n" + state.description + "\n" + state.inventory
print(observation)

                    ________  ________  __    __  ________        
                   |        \|        \|  \  |  \|        \       
                    \$$$$$$$$| $$$$$$$$| $$  | $$ \$$$$$$$$       
                      | $$   | $$__     \$$\/  $$   | $$          
                      | $$   | $$  \     >$$  $$    | $$          
                      | $$   | $$$$$    /  $$$$\    | $$          
                      | $$   | $$_____ |  $$ \$$\   | $$          
                      | $$   | $$     \| $$  | $$   | $$          
                       \$$    \$$$$$$$$ \$$   \$$    \$$          
              __       __   ______   _______   __        _______  
             |  \  _  |  \ /      \ |       \ |  \      |       \ 
             | $$ / \ | $$|  $$$$$$\| $$$$$$$\| $$      | $$$$$$$\
             | $$/  $\| $$| $$  | $$| $$__| $$| $$      | $$  | $$
             | $$  $$$\ $$| $$  | $$| $$    $$| $$      | $$  | $$
             | $$ $$\$$\$$| $$  | $$| $$$$$$$\| $$      | $$  | $$
             | $$$$  \$$$$| $$__/ $$| $$  | $$| $$_____ | $$__/ $$
             | $$$    \$$$ \$$    $$| $$  | $$| $$     \| $$    $$
              \$$      \$$  \$$$$$$  \$$   \$$ \$$$$$$$$ \$$$$$$$ 

Hey, thanks for coming over to the TextWorld today, there is something I need you to do for me. First of all, go east. After that, take the passkey from the trunk inside the laundry place. And then, unlock the box in the laundry place with the passkey. And once you've done that, you win!

-= Kitchen =-
You arrive in a kitchen. A standard one. I guess you better just go and list everything you see here.

You see an opened chest right there by you. The chest is empty! This is the worst thing that could possibly happen, ever!

There is a closed door leading south. There is an unguarded exit to the east.

-= Kitchen =-
You arrive in a kitchen. A standard one. I guess you better just go and list everything you see here.

You see an opened chest right there by you. The chest is empty! This is the worst thing that could possibly happen, ever!

There is a closed door leading south. There is an unguarded exit to the east.
You are carrying: a top hat.

after dropping the top hat:

You drop the top hat on the ground.

-= Kitchen =-
You arrive in a kitchen. A standard one. I guess you better just go and list everything you see here.

You see an opened chest right there by you. The chest is empty! This is the worst thing that could possibly happen, ever!

There is a closed door leading south. There is an unguarded exit to the east.

There is a top hat on the floor.
You are carrying nothing.

NB: the state.feedback will be almost always the same as state.description when you enter a room (i.e., after performing a go action). You can also see in the first observation they both contain the initial room description at the first step.

junkyul commented 2 years ago

Thanks very much!

From the documentation, description is the output of the look command. Does it mean Textworld internally calls the look command after the command given by an agent? Does this description have a template that translates the internal game state to the text format?

MarcCote commented 2 years ago

The current version of TextWorld relies on the compiled game (through Inform7) to generate text (including commands' feedback). This means that TextWorld does indeed issue a look command under the hood but is not counted towards the number of moves taken. Same thing for inventory.

Does this description have a template that translates the internal game state to the text format?

The AlfWorld version of TextWorld does have such a template but it is quite limited.

junkyul commented 2 years ago

Thanks very much for your answer!