example on finetuning Llama2-70b model on multiple nodes

[giaosudau]

I am finding an example on how to finetuning Llama2-70b model on multiple nodes

[concretevitamin]

This example on finetuning Llama on 1 node is a probably good starting point:

• https://github.com/skypilot-org/skypilot/tree/master/llm/vicuna-llama-2
• https://github.com/skypilot-org/skypilot/blob/master/llm/vicuna-llama-2/train.yaml

[github-actions[bot]]

This issue is stale because it has been open 120 days with no activity. Remove stale label or comment or this will be closed in 10 days.

[romilbhardwaj]

Bumping this - we should add multi-node fine-tuning examples for the llama model family.