[fp16 error] In GPT-2 run example

[lin88lin8850]

when I set fp16=true during training gpt-2, error happens:

[TXacs]

This error come from the gpt2 model of transformers. Because the reorder_and_upcast_attn is True in config.

[lin88lin8850]

thanks a lot! bug solved, the issue will be closed