search

open-mmlab / mmcv

OpenMMLab Computer Vision Foundation
https://mmcv.readthedocs.io/en/latest/
Apache License 2.0
5.91k stars 1.65k forks source link

[Feature Request] Apex support for mixed precision training #1719

Closed nijkah closed 2 years ago

nijkah commented 2 years ago

Describe the feature

Motivation Many recent research works are based on apex for mixed precision training.

So I want to add this feature to mmcv with the name of class EpochBasedRunnerApex. I am thinking implementation of just wrapping EpochBasedRunner as in Swin-Transformer Implementation.

Is it good idea to mmcv? Then I will create the PR.

Related resources https://github.com/SwinTransformer/Swin-Transformer-Object-Detection https://github.com/facebookresearch/ConvNeXt https://github.com/microsoft/unilm/tree/master/beit/semantic_segmentation

HAOCHENYE commented 2 years ago

Thanks for your advice, but what is the advantage of apex over torch.cuda.amp? mmcv implements mixed precision training via Fp16OptimizerHook.

nijkah commented 2 years ago

@HAOCHENYE From this link, it says Apex is a first implementation of amp, and deprecated. So, it doesn't seem to need to support that. I think I have to check why these works used apex for amp.