Multi GPU Training | Vertical Scaling

BharatSahAIyak / autotune

A comprehensive toolkit for seamless data generation and fine-tuning of NLP models, all conveniently packed into a single block.

MIT License

9 stars 5 forks source link

Open TakshPanchal opened 4 months ago

TakshPanchal commented 4 months ago

Right now autotune supports only horizontal scaling. For training large models we have to use multiple GPUs and do distributed training.

let's discuss @Ansh-Sarkar in free times as this is a low priority ticket right now.

cc: @GautamR-Samagra @singhalkarun