search

eloialonso / iris

Transformers are Sample-Efficient World Models. ICLR 2023, notable top 5%.
https://openreview.net/forum?id=vhFu1Acb0xb
GNU General Public License v3.0
804 stars 80 forks source link

Compatibility of continuous action space #16

Closed dibbla closed 1 year ago

dibbla commented 1 year ago

Thanks for this great work and thanks for opensourcing it!

As I can see in the paper, it shows experiment result in discrete action space but not continuous one. And as mentioned in #13 by @2M-kotb, "the transformer-based world model expects action tokens in discrete form". So, is it possible to modify IRIS for continous actions?

If it's possible, do you have any suggestion working on this?

Thanks!

vmicheli commented 1 year ago

Hey, thanks for the kind words!

Apart from what was discussed in #13, you could try to map continuous actions to the input dimension of the Transformer via a linear layer. As the world model does not predict actions, action tokens do not have to be discrete.

Hope that helps!

dibbla commented 1 year ago

Hey, thanks for the kind words!

Apart from what was discussed in #13, you could try to map continuous actions to the input dimension of the Transformer via a linear layer. As the world model does not predict actions, action tokens do not have to be discrete.

Hope that helps!

Thanks!