Closed renmada closed 3 years ago
Compare with fp32 finetuning , It takes about 10x more time to inference dev data during training when do Integer-only finetune to Integer-only finetuning. How can I do INT8 inference and achieve the seepup as described in paper?
Compare with fp32 finetuning , It takes about 10x more time to inference dev data during training when do Integer-only finetune to Integer-only finetuning. How can I do INT8 inference and achieve the seepup as described in paper?