search

OpenGVLab / EfficientQAT

EfficientQAT: Efficient Quantization-Aware Training for Large Language Models
224 stars 17 forks source link

DATA FOR TRAINING #12

Closed LiMa-cas closed 3 months ago

LiMa-cas commented 3 months ago

hi，in the paper you said “we use 4096 samples from RedPajama with a context length of 2048”， is it enough for QAT?

ChenMnZ commented 3 months ago

Hi, you can refer Figure 4 and Table 5 in paper for the ablation of training samples.