search

Tushar-N / pytorch-resnet3d

I3D Nonlocal ResNets in Pytorch
245 stars 39 forks source link

Question about the implementation of Non-local block #4

Closed szx503045266 closed 5 years ago

szx503045266 commented 5 years ago

Hi. I didn't see the code below in other implemetation version of non-local block and I don't quite understand its usage. theta_phi_sc = theta_phi * (self.dim_inner**-.5) https://github.com/Tushar-N/pytorch-resnet3d/blob/e997fc6a4a632502460aece5e83e0bd2fd6206d9/models/resnet.py#L114 Could you please explain what it is used for? Thanks a lot.

Tushar-N commented 5 years ago

This is the scale operation as described in Sec 3.2.1 of this paper. It isn't described in the original NL paper, but it shows up in the official authors implementation here:

theta_phi_sc = model.Scale(theta_phi, theta_phi, scale=dim_inner**-.5)

cfg.NONLOCAL.USE_SCALE is set to True in the config that was used to generate the pretrained models as seen in the training logs.

This comment on the original repo explains some other differences from common implementations. This repo just converts the c2 model to pytorch (rather than train from scratch), so the model definitions had to be consistent.

szx503045266 commented 5 years ago

Thanks for your quick and detailed reply! It helps me a lot.