Visualize a policy from a checkpoint

[hai-h-nguyen]

Hello,

Is it supported to visualize a trained policy from a checkpoint? If yes, how would I do that? I saw there was a tutorial on how to evaluate a checkpoint, but I want to visually inspect the learned policy too.

[Lucaweihs]

Hi @hai-h-nguyen,

This definitely should be possible depending on the type of visualization you're looking for. A few quick questions:

1. What environment are you training on?
2. What type of visualizations do you want? E.g.
   • Do you want to run your model on a single task in a loop just to get a sense of what it's doing?
   • Do you want to run your model on a whole bunch of episodes, save the results (e.g. to tensorboard or to videos on disk), and then look through them after the fact?

Some initial pointers:

• We have a visualization tutorial as well that shows you how to visualize episodes in tensorboard. It's a bit more involved though as you need to have some idea of how things are being stored in AllenAct.
• If you look at this file you can see how you can use the InferenceAgent class to run your model on a task in python while avoiding all the AllenAct training/evaluation infrastructure. Note that you'd need to pass the path to your model checkpoint in the call to InferenceAgent.from_experiment_config(...) as currently it just runs with a randomly initialized agent. Once you have this running you can visualize observations returned from the task however you like.
• @mattdeitke recently made a PR #339 which adds more functionality to AllenAct and lets you set up custom logging scripts. He's used this to, for instance, log from AllenAct to wand.ai.

[hai-h-nguyen]

Hi, thanks for a quick reply.

1. I am training in my custom environments.
2. I am looking for the first option - to get a sense of what it's doing over a number of episodes. I also wish to make a video later but I can just use a screen recording software for that purpose.
3. Thanks I will look into those pointers first.

[hai-h-nguyen]

Thanks, I am able to visualize the policy using InferenceAgent.