search

artidoro / qlora

QLoRA: Efficient Finetuning of Quantized LLMs
https://arxiv.org/abs/2305.14314
MIT License
9.74k stars 800 forks source link

Why do we print just half of `trainable_params" when using 4-bits? #260

Open HanGuo97 opened 10 months ago

HanGuo97 commented 10 months ago

Hi, first of all, thanks for this amazing repo!

I have a quick (and very likely dumb) question about the following line. Specifically, why do we print just half of the variable when bits==4? https://github.com/artidoro/qlora/blob/7f4e95a68dc076bea9b3a413d2b512eca6d004e5/qlora.py#L418

Thanks in advance for your time!