MultiGPU support or better intergration for loading models

huggingface / setfit

Efficient few-shot learning with Sentence Transformers

https://hf.co/docs/setfit

Apache License 2.0

2.24k stars 223 forks source link

MultiGPU support or better intergration for loading models #509

Open drewskidang opened 7 months ago

drewskidang commented 7 months ago

Is it posssible to train on more than one gpu https://huggingface.co/BAAI/bge-m3 currently ooming on this model. BGE allows to load fp16 but cant do it with sefit

H-Ismael commented 5 months ago

Can this also cause the OutOfMemoryError: CUDA out of memory. Tried to allocate ... ?