Adding action space statistics

[pranavguru]

This PR includes the addition of code to calculate action space statistics across an entire dataset, and use this information in the action description of the prompt as a fallback for datasets that do not have verbal descriptions of the action space.

Changes:

prompt.py:

• Include the new action category for the prompt (continuous action spaces without verbal descriptions)

openx_dataloader.py:

• Track the action space statistics as a one-time calculation for each dataset, when the PyTorch dataloader intrinsically calls the len function
• Function to return the action statistics
• Return the OpenXDataset object along with the dataloader in order to access the action stats

openx_module.py:

• Obtains the action states before creating the prompt
• Checks the action description to see if there is a placeholder (currently following @KickItLikeShika's placeholder format) that indicates the absence of a verbal description --> creates a an action space description tuple based on the statistics --> passes this new description to the prompt formatting function

[devjwsong]

Not sure how it works. So if there is no verbal description on the action space, GPT-4 should generates the actions only with the statistics? How does the model know what each action dimension represent for?

[pranavguru]

Not sure how it works. So if there is no verbal description on the action space, GPT-4 should generates the actions only with the statistics? How does the model know what each action dimension represent for?

While this is definitely not ideal (and yes, the model will not conceptually know what each action dimension represents), this is just a fallback for the cases where we are not able to find verbal descriptions for the action spaces. This is just a backup method we discussed in the last meeting if you recall - this might be the only other option if we are not able to find verbal descriptions of the action space.

[harshsikka]

this seems reasonable as a backup plan if we aren't able to surface the real details of the action space. Of course we have to get as many details as possible, which @KickItLikeShika and @pranavguru are working on i think

[devjwsong]

Okay, if that's the case, that's fine for now. I think that using GPT-4 itself might not be just a thought, it can actually be the direct measure to this problem if we cannot find the action space description. We can think about it later.