SyncBN with `norm_eval=True`

[liming-ai]

Prerequisite

• [X] I have searched Issues and Discussions but cannot get the expected help.
• [X] I have read the FAQ documentation but cannot get the expected help.
• [X] The bug has not been fixed in the latest version (master) or latest version (3.x).

💬 Describe the reimplementation questions

I just realized that SyncBN should set norm_eval=False. However, there is nothing to highlight this in the docs or elsewhere. Should we set a hook to check that?

If we use SyncBN and norm_eval=True at the same time, does it mean that it is equivalent to using BN and norm_eval=True because the value of BN has not been modified during training?

Also, using SyncBN(requires_grad=True) and norm_eval=True at the same time, will it hurt performance?

Environment

It is not relative to the environment.

Expected results

No response

Additional information

No response

[cskkxjk]

I am also confused by the syncbn setting as well!

[liming-ai]

Hi @ZwwWayne,

Could you please check this?