jbloomAus / DecisionTransformerInterpretability

Interpreting how transformers simulate agents performing RL tasks
https://jbloomaus-decisiontransformerinterpretability-app-4edcnc.streamlit.app/
MIT License
68 stars 16 forks source link

Write a Probe Environment that tests a models ability to look at previous observations #31

Open jbloomAus opened 1 year ago

jbloomAus commented 1 year ago

We currently have 5 probe environments for single timestep models and I'd like a prob environment to test if a model can learn:

  1. to take the correct action as a function of a previous observation
  2. to test that the model doesn't learn the correct action if it's outside the recurrence range or something (more speculative)
jbloomAus commented 1 year ago

This turns out to be hard because we get underflow with the prob environments on the RNN and need to edit the trajectory LSTM model which expects to get mini-grid environment frames. I will need to think more about how to test it with probe environments.