search

ikostrikov / pytorch-a2c-ppo-acktr-gail

PyTorch implementation of Advantage Actor Critic (A2C), Proximal Policy Optimization (PPO), Scalable trust-region method for deep reinforcement learning using Kronecker-factored approximation (ACKTR) and Generative Adversarial Imitation Learning (GAIL).
MIT License
3.53k stars 832 forks source link

Combine Acktr model with grad-cam #268

Open seed851218 opened 3 years ago

seed851218 commented 3 years ago

I am new on this code, My question is that if i want to get the grad on the CNN layer by using register_backward_hook function. Should I remove the code "with torch.no_grad():" when training this model? I modify the other grad-cam code but still get bug when I call "register_backward_hook" function. Is there any one can help me or any example? My English is not good, Thank u!

seed851218 commented 3 years ago

this is work image

but code didnt get in this function image

This is my foward function image

This is my backward function image

on the enjoy script , if I remove the code "with torch.no_grad():" that it gives me the bug:

Traceback (most recent call last): File "enjoy.py", line 139, in cam = gcam(state,recurrent_hidden_states,masks) File "/media/seed/C2C6AED6C6AEC9CD/AirSim/PythonClient/multirotor/pytorch-a2c-ppo-acktr-gail/pure_code/pytorch-a2c-ppo-acktr-gail/grad_campp.py", line 123, in call output, , index, , = self.net.act_test(inputs, h_0, c_0,deterministic = True) # [1,num_classes] File "/media/seed/C2C6AED6C6AEC9CD/AirSim/PythonClient/multirotor/pytorch-a2c-ppo-acktr-gail/pure_code/pytorch-a2c-ppo-acktr-gail/a2c_ppo_acktr/model.py", line 55, in act_test value, actor_features, rnn_hxs = self.base(inputs, rnn_hxs, masks) File "/home/seed/.local/lib/python3.6/site-packages/torch/nn/modules/module.py", line 722, in _call_impl result = self.forward(*input, kwargs) File "/media/seed/C2C6AED6C6AEC9CD/AirSim/PythonClient/multirotor/pytorch-a2c-ppo-acktr-gail/pure_code/pytorch-a2c-ppo-acktr-gail/a2c_ppo_acktr/model.py", line 207, in forward x = self.main(inputs / 255.0) File "/home/seed/.local/lib/python3.6/site-packages/torch/nn/modules/module.py", line 722, in _call_impl result = self.forward(*input, *kwargs) File "/home/seed/.local/lib/python3.6/site-packages/torch/nn/modules/container.py", line 117, in forward input = module(input) File "/home/seed/.local/lib/python3.6/site-packages/torch/nn/modules/module.py", line 722, in _call_impl result = self.forward(input, kwargs) File "/media/seed/C2C6AED6C6AEC9CD/AirSim/PythonClient/multirotor/pytorch-a2c-ppo-acktr-gail/pure_code/pytorch-a2c-ppo-acktr-gail/a2c_ppo_acktr/algo/kfac.py", line 82, in forward x = self.module(input) File "/home/seed/.local/lib/python3.6/site-packages/torch/nn/modules/module.py", line 714, in _call_impl result = hook(self, input) File "/media/seed/C2C6AED6C6AEC9CD/AirSim/PythonClient/multirotor/pytorch-a2c-ppo-acktr-gail/pure_code/pytorch-a2c-ppo-acktr-gail/a2c_ppo_acktr/algo/kfac.py", line 144, in _save_input if torch.is_grad_enabled() and self.steps % self.Ts == 0: AttributeError: 'KFACOptimizer' object has no attribute 'steps'

can anyone help me? Thank u so much.

@ikostrikov Need ur help. I stuck on this for a long time.

sungreong commented 2 years ago

did you solve this?