jbloomAus / DecisionTransformerInterpretability

Interpreting how transformers simulate agents performing RL tasks
https://jbloomaus-decisiontransformerinterpretability-app-4edcnc.streamlit.app/
MIT License
61 stars 15 forks source link

Check how LSTM model BOW init is being done and whether it needs a fix #62

Open jbloomAus opened 1 year ago

jbloomAus commented 1 year ago

The BOWEmbedding from BabyAI has very large vectors on init. I wonder if that's really terrible and was slowing down my training of demo generating models. Need to investigate at some point.