search

facebookresearch / LLM-QAT

Code repo for the paper "LLM-QAT Data-Free Quantization Aware Training for Large Language Models"
Other
251 stars 24 forks source link

Is there an efficient way to generate data? #14

Closed benyang0506 closed 1 year ago

benyang0506 commented 1 year ago

Hello, I'm trying to run the data generation. But the data generation is particularly slow right now. Is there an efficient way to generate data? Thanks!

wejoncy commented 1 year ago

Move model and input to GPU will imrpove it dramaticlly

benyang0506 commented 1 year ago

Thank you for your answer! I noticed that you used 64 gpu to generate data, I wonder how long it took.

liuzechun commented 1 year ago

Updated the code. It takes approximately 1 day to generate 100k sentences.