OpenGVLab / VideoMAEv2

[CVPR 2023] VideoMAE V2: Scaling Video Masked Autoencoders with Dual Masking
https://arxiv.org/abs/2303.16727
MIT License
524 stars 63 forks source link

The parameter grad_norm appears to be inf and then nan when input resolution is 112*112 during the pre-training on VIT-Small backbone #53

Open DragonWang-cell opened 10 months ago

DragonWang-cell commented 10 months ago

Hello, thank you very much for your significant contribution to the computer vision community! When I set my input resolution to 112*112 and do the pre-training on VIT-Small backbone the parameter grad_norm appears to be inf and then nan and then back to normal, is this normal or abnormal? If the training is abnormal what should I do to avoid this, looking forward and thanking you for your answer! bec87fed107a42d2df8ca25b5d993c5 2feaecdc2dec37011d1bb8d5baebbca

congee524 commented 8 months ago

I think the training is normal. How's it going back then?