search

lucidrains / CoCa-pytorch

Implementation of CoCa, Contrastive Captioners are Image-Text Foundation Models, in Pytorch
MIT License
1.04k stars 88 forks source link

Extractor in vit_pytorch will detach the tensor. #6

Closed techkang closed 2 years ago

techkang commented 2 years ago

Thanks for your code! I think I may find a little bug. The cloned tensor in Extractor will be detached (https://github.com/lucidrains/vit-pytorch/blob/main/vit_pytorch/extractor.py#L39). So gradient may not propagate back to image encoder.

lucidrains commented 2 years ago

@techkang Hi Kang! Yes you are right, should be fixed in 0.35.8 of vit-pytorch! https://github.com/lucidrains/CoCa-pytorch/commit/d70417d5fbee22001d25fc35a89f54cacde8b36a

techkang commented 2 years ago

The bug is fixed. Thank you.