Open XavierXiao opened 1 year ago
should be fixed now
Thanks! One interesting thing I found is fp 16 seems to use similar or even more GPU memory. Both fp16 and fp32 can train with bs=8 on a GPU with 48 GB ram, and when bs = 2, I even found fp32 takes less memory.
It seems like there is a bug when setting use_fp16 = False, the log is