When we generate agent-environment rollouts that maximally activate agent's hidden state neurons in the directions of the PCs, we find that the visualisations look more meaningful. Here we find directions that appear to correspond to XYZ.
Same 3 panel figure as above (has obs, pca, and barchart, no saliency of obs or on barchart) but for samples that maximise each of the PCs (or NMFs if it happens to be what you use for the bar chart)
where above means: "Figure that depicts pca plot with option for which PCs to choose. 3 panels. It's a video of the agent observations (no saliency) and the hx as it moves through pca space with bar chart (no saliency either). "
Corresponding text in draft: