difference between two GPT models used in this repo?

kzl / decision-transformer

Official codebase for Decision Transformer: Reinforcement Learning via Sequence Modeling.

MIT License

2.38k stars 450 forks source link

difference between two GPT models used in this repo? #17

Closed ChenDRAG closed 3 years ago

ChenDRAG commented 3 years ago

Hi, thanks a lot for releasing the code. I noticed that you use separate GPT models in atari and mujoco. One comes from minGPT and the other comes from hugging face(I'm not sure, which specific file of hugging face repo, could you tell?).
I'm wondering is there any difference between the two GPT models above? I would like to run experiment of Gym Mujoco, but the code in /gym/decision_transformer/models/decision_transformer.py seems scary. Is it possible that I use minGPT implementation?

lili-chen commented 3 years ago

I think you should be able to adapt the minGPT implementation to Gym, but you'd have to change the details (e.g., to handle continuous action space) which could take some time. There are some very slight implementation differences in the models (e.g., position embedding is handled a bit differently) but they shouldn't affect the results much.