search

microsoft / IBAC-SNI

Code to reproduce the NeurIPS 2019 paper "Generalization in Reinforcement Learning with Selective Noise Injection and Information Bottleneck" by Maximilian Igl, Kamil Ciosek, Yingzhen Li, Sebastian Tschiatschek, Cheng Zhang, Sam Devlin and Katja Hofmann.
https://arxiv.org/abs/1910.12911
Other
52 stars 17 forks source link

Evaluate and visualize scripts don't seem to work #2

Closed sedand closed 4 years ago

sedand commented 4 years ago

Hi there, I've run into a problem running the evaluate.py and visualize.py scripts (using models trained on the gridworld).

Maybe it is a misunderstanding on my side, on how these scripts are meant to be used, but they don't seem to be "compatible" with the kind of ACModel (IBAC-SNI/torch_rl/model.py) saved after a training, as they don't contain the required attributes (e.g. recurrent) or methods (forward(..) )

Steps to reproduce: 1: Train a gridworld model as stated in the readme:

python -m scripts.train --frames 100000000 --algo ppo --env MiniGrid-MultiRoom-N3r-v0 --model N3r-vib1e6 --save-interval 100 --tb --fullObs --model_type default2 --use_bottleneck --beta 0.000001

2: Run the evaluation script on the result folder

python -m scripts.evaluate --env MiniGrid-MultiRoom-N2r-v0 --model N3r-vib1e6

Error message:

Traceback (most recent call last):
  File "/home/andreas/miniconda2/envs/ibac/lib/python3.7/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/home/andreas/miniconda2/envs/ibac/lib/python3.7/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/home/andreas/research/others/IBAC-SNI/torch_rl/scripts/evaluate.py", line 51, in <module>
    agent = utils.Agent(args.env, env.observation_space, model_dir, args.argmax, args.procs)
  File "/home/andreas/research/others/IBAC-SNI/torch_rl/utils/agent.py", line 17, in __init__
    if self.acmodel.recurrent:
  File "/home/andreas/miniconda2/envs/ibac/lib/python3.7/site-packages/torch/nn/modules/module.py", line 576, in __getattr__
    type(self).__name__, name))
AttributeError: 'ACModel' object has no attribute 'recurrent'

Thanks for your help!

maximilianigl commented 4 years ago

Thanks for pointing this out. I've never used the evaluate script (which came with the original torch_rl package), so I didn't adapt it to the changes I made to the agent. I'll look into it on the weekend. If you need it urgently, it might work if you just take out all the references to memories and recurrent from the /torch_rl/utils/agent.py file.

sedand commented 4 years ago

Ah, I see. I thought this script was used to generate the middle plot of Figure 2 in the paper (success rate VS nr. rooms). The right plot (return VS frames) of Figure 2 can be reproduced using the plots.py script, correct? Thanks a lot for your help & time!

maximilianigl commented 4 years ago

I've fixed evaluate.py (which just runs one agent and tests how well it does) and visualize.py (which renders one agent), at least they're working for me now. Please let me know if not for you.

I also added detailed_multiroom_results.py, which I used to generate the success rate vs. nr rooms plot. Thanks for pointing out it was missing - It wasn't part of the repo yet because I did it after I had finished the internship during which the project was done.

And yes plots.py produces the right plot. I'll close the issue for now but please let me know if you have any questions or something's still not working!

maximilianigl commented 4 years ago

(be sure to also pass --fullObs to the visualize and evaluate scripts if the agent was trained on the fully observable environment. For the detailed_multiroom_results.py there's a corresponding flag in the script.)