Open SZ-ing opened 3 weeks ago
Take a look at these tutorials:
There are many tools out there to help convert a model to one that can run on multiple GPUs that can help automate this: https://www.deepspeed.ai/tutorials/automatic-tensor-parallelism/ https://huggingface.co/docs/accelerate/basic_tutorials/launch
❓ Question
What you have already tried
Environment
conda
,pip
,libtorch
, source):Additional context
As the title, I have a machine with multiple GPUs and I would like to know if there is any way to evenly distribute the model across these GPUs. Is there any way to achieve this?