adaptor_v2.py should support multi gpu mode.

Lightning-AI / lit-llama

Implementation of the LLaMA language model based on nanoGPT. Supports flash attention, Int8 and GPTQ 4bit quantization, LoRA and LLaMA-Adapter fine-tuning, pre-training. Apache 2.0-licensed.

Apache License 2.0

5.97k stars 518 forks source link

adaptor_v2.py should support multi gpu mode. #455

Closed MartinForReal closed 9 months ago