Why can directly use FP16 during training?

[shoutOutYangJie]

I find in your code, you use FP16 for the last 4 layers for stylegan-2. However, you don't activate "torch.cuda.amp"

[nurpax]

We don't use AMP to automatically turn on FP16 for layers, rather we use FP16 manually. Search for use of "float16" in the stylegan3 codebase.