search

mazpie / choreographer

[ICLR 2023] Choreographer: a model-based agent that discovers and learns unsupervised skills in latent imagination, and it's able to efficiently coordinate and adapt the skills to solve downstream tasks.
https://skillchoreographer.github.io/
MIT License
36 stars 5 forks source link

Choreographer in discrete action spaces #1

Open swy99 opened 1 year ago

swy99 commented 1 year ago

I am trying to run Choreographer in environments with discrete action spaces.

The implementation was done with the help of the part of DreamerV2 code that are responsible for discrete action space. But my implementation did not work very well, especially lbs shows somewhat collapsing of actions, which means that it selects only one action in pixel based environments.

This link shows a part of the changes I have made besides adding some env wrappers to envs.py: https://github.com/swy99/choreographer_discrete/commit/e27b256099f6907122999e9e1bc1efbe0b090c0f

I have mainly tried MiniGrid with different sets of hyperparameters but all my attempts failed.

Did you have any chances to try discrete action space environments? Could you share some working sets of hyperparameters?

mazpie commented 1 year ago

Hi, I adapted this codebase to several discrete action space environments in the past (Atari, Habitat, Unity environments), and it worked fine, with the same hyperparameters you can find in the current configs files, but to be fair I haven't tested it on MiniGrid yet.

I had a look at the changes you made and they look sensible. Perhaps, there is some issue with the world model's training? Are you using pixel-based inputs (64x64x3) or are you testing on the low-dimensional Minigrid inputs?

swy99 commented 1 year ago

Thanks a lot for taking your time to respond. I am currently using pixel-based inputs (64x64x3).