What is the best practice for me to add mixed precision training in mmaction2

[weiaicunzai]

Hi, guys. thanks for your great work. I am new to mmaction2, and I have noticed that FP16 training is not fully supported (e.g. https://github.com/open-mmlab/mmaction2/issues/1993, https://github.com/open-mmlab/mmaction2/issues/1510 , https://github.com/open-mmlab/mmaction2/issues/1769) as other open-mmlab libraries (e.g. mmsegmentation).

So, could you please tell me what is the best practice for me to use FP16 training in my own project ? Thanks in advance.

[hukkai]

@weiaicunzai My suggestion is not to use mixed precision for 3d networks as there can be a performance drop with unknown reasons: https://github.com/pytorch/pytorch/issues/63313. I have also experienced related issues.

[weiaicunzai]

@weiaicunzai My suggestion is not to use mixed precision for 3d networks as there can be a performance drop with unknown reasons: pytorch/pytorch#63313. I have also experienced related issues.

Thanks for your reply, Does "3D networks" include 3D Transformers? Or simply 3D CNNs? Thanks.

[hukkai]

3D CNNs. Not sure for the 3D transformer case.

[weiaicunzai]

3D CNNs. Not sure for the 3D transformer case.

Thanks.