search

ContextualAI / gritlm

Generative Representational Instruction Tuning
https://arxiv.org/abs/2402.09906
MIT License
538 stars 39 forks source link

[Fix] Don't use model .to( device when using bitsandbytes quant #37

Closed kabachuha closed 4 months ago

kabachuha commented 4 months ago

As bitsandbytes send the model automatically to the available devices, and using .to with models will cause an error