AgentInterpretation

[josiahls]

Discussion ClassificationInterpretation is one of the greatest reasons to use the fastai library. I believe that a similar AgentInterpretation class could single handedly turn this repo into a "hey this spaghetti code is actually useful" as opposed to the current "this spaghetti code needs some more meatballs still :( ".

Some of the questions I have is:

• [x] Functions important for the AgentInterpretation object.
• [x] Have a plot_top_episode which returns a sequence of frames. This should make agent evaluation easier for jupyter notebooks.
• [x] Have a similar plot_multi_top_episodes.
• [x] Have an easy save to GIF?

Most important

• [x] Have heatmap_rewards. Have some function to show a heat map of where the highest and lowest rewards are being distributed. How do we plan to do this?
  • Run only on images? Show matrix of non-image state spaces? For example, the gym_maze should be easy to heatmap which is why it is serving as my bug test env for my discrete agents.

Edit [1]: Added a heatmap rewards function and a rewards plotter. The heatmap rewards function will only work for grid like envs where the state space is 2 dimensional (2D maze / grid). I am thinking about how to extend this since heatmapping rewards can be one of the most effective ways of debugging RL agents. For now, I have a function for testing Discrete agents. I will want to add continuous heatmapping somehow.

• [x] Add continuous heatmapping somehow. AgentInterpretation has been renamed AgentInterpretationv1. This is because I think that I want to make a new object AgentInterpretation that will be completely different possibly in function, but want to back compatability / always have a tool that can allow for debugging.

[josiahls]

Rerference docs_src/rl.core.mdp_interpreter.ipynb. Can now do reward output, but also single episode interactive gif:

Allows you to easily investigate a single episode within 2 lines. Will soon convert it to one line, however at the moment the cell doesnt register the player unless the ipython_display is executed in the cell directly.