Closed harsham05 closed 5 years ago
loading a saved state dict into a model will default to whatever type the model is declared as in your code. The loss of precision going from fp32->fp16 is insignifcant. However, fp16 math is really emulated fp32 math on cpu, so it's even slower than fp32. I would not recommend using fp16 on cpu.
Do the fp32 pre-trained models default to fp16 when run on CPU? Is the loss of precision when running on the CPU significant such that its not recommended?