search

facebookresearch / SlowFast

PySlowFast: video understanding codebase from FAIR for reproducing state-of-the-art video models.
Apache License 2.0
6.5k stars 1.2k forks source link

Which layer should I use for visualizing by Grad-CAM? #314

Open Shumpei-Kikuta opened 3 years ago

Shumpei-Kikuta commented 3 years ago

Thank you for your work! I am impressed with the integration of the visualization technique GradCAM!

I am currently applying GradCAM to Kinetics. I am wondering which layer I should use for visualization. When I read Grad-CAM paper, the author recommended that we generally use the last layer before the fully connected layer.

Could you tell me which layer we should use? Also, could you provide me which specific layer name we should use?

https://github.com/facebookresearch/SlowFast/blob/fd41618191d3c21c1ca21a61369ce9917646cf9c/slowfast/config/defaults.py#L655

guojingbuaa commented 3 years ago

I meet the same problem。 I print all the layer names of X3D_L ：s4.pathway0_res20.branch2.b_bn.running_var s4.pathway0_res20.branch2.b_bn.num_batches_tracked s4.pathway0_res20.branch2.se.fc1.weight s4.pathway0_res20.branch2.se.fc1.bias s4.pathway0_res20.branch2.se.fc2.weight s4.pathway0_res20.branch2.se.fc2.bias s4.pathway0_res20.branch2.c.weight s4.pathway0_res20.branch2.c_bn.weight s4.pathway0_res20.branch2.c_bn.bias s4.pathway0_res20.branch2.c_bn.running_mean s4.pathway0_res20.branch2.c_bn.running_var s4.pathway0_res20.branch2.c_bn.num_batches_tracked s4.pathway0_res21.branch2.a.weight s4.pathway0_res21.branch2.a_bn.weight s4.pathway0_res21.branch2.a_bn.bias s4.pathway0_res21.branch2.a_bn.running_mean s4.pathway0_res21.branch2.a_bn.running_var s4.pathway0_res21.branch2.a_bn.num_batches_tracked s4.pathway0_res21.branch2.b.weight s4.pathway0_res21.branch2.b_bn.weight s4.pathway0_res21.branch2.b_bn.bias s4.pathway0_res21.branch2.b_bn.running_mean s4.pathway0_res21.branch2.b_bn.running_var s4.pathway0_res21.branch2.b_bn.num_batches_tracked s4.pathway0_res21.branch2.c.weight s4.pathway0_res21.branch2.c_bn.weight s4.pathway0_res21.branch2.c_bn.bias s4.pathway0_res21.branch2.c_bn.running_mean s4.pathway0_res21.branch2.c_bn.running_var s4.pathway0_res21.branch2.c_bn.num_batches_tracked s4.pathway0_res22.branch2.a.weight s4.pathway0_res22.branch2.a_bn.weight s4.pathway0_res22.branch2.a_bn.bias s4.pathway0_res22.branch2.a_bn.running_mean s4.pathway0_res22.branch2.a_bn.running_var s4.pathway0_res22.branch2.a_bn.num_batches_tracked s4.pathway0_res22.branch2.b.weight s4.pathway0_res22.branch2.b_bn.weight s4.pathway0_res22.branch2.b_bn.bias s4.pathway0_res22.branch2.b_bn.running_mean s4.pathway0_res22.branch2.b_bn.running_var s4.pathway0_res22.branch2.b_bn.num_batches_tracked s4.pathway0_res22.branch2.se.fc1.weight s4.pathway0_res22.branch2.se.fc1.bias s4.pathway0_res22.branch2.se.fc2.weight s4.pathway0_res22.branch2.se.fc2.bias s4.pathway0_res22.branch2.c.weight s4.pathway0_res22.branch2.c_bn.weight s4.pathway0_res22.branch2.c_bn.bias s4.pathway0_res22.branch2.c_bn.running_mean s4.pathway0_res22.branch2.c_bn.running_var s4.pathway0_res22.branch2.c_bn.num_batches_tracked s4.pathway0_res23.branch2.a.weight s4.pathway0_res23.branch2.a_bn.weight s4.pathway0_res23.branch2.a_bn.bias s4.pathway0_res23.branch2.a_bn.running_mean s4.pathway0_res23.branch2.a_bn.running_var

and _C.TENSORBOARD.MODEL_VIS.LAYER_LIST = ["head.conv_5.weight"] get error can you help me with this question ? many thanks

Shumpei-Kikuta commented 3 years ago

@guojingbuaa I solved this problem by writing as follows:

_C.TENSORBOARD.MODEL_VIS.LAYER_LIST: ['s5/pathway1_res2', 's5/pathway0_res2']

But in my case, I used Slowfast Network, which you didn't use. Can you show me the error message?

ljorquera commented 3 years ago

hi @Shumpei-Kikuta I am trying to make GradCAM but had many issues so far. It seems that you have been able to use it, could you point me to the documentation you used (I did not find much). Also if you documented somewhere your configuration, that would be a wonderful resource. Thanks!