Question: How DLRover integrate with Llama Factory? - Githubissues

intelligent-machine-learning / dlrover

DLRover: An Automatic Distributed Deep Learning System

Other

1.21k stars 149 forks source link

Question: How DLRover integrate with Llama Factory? #1244

Open hetingyou opened 1 month ago

hetingyou commented 1 month ago

直觉是修改examples/pytorch/nanogpt/elastic_job.yaml:

command:

/bin/bash
-c
"dlrover-run --network-check --nnodes=$NODE_NUM \ --nproc_per_node=1 --max_restarts=1 \ ./examples/pytorch/nanogpt/train.py \ --data_dir /data/nanogpt/"

改为如下形式，报错：找不到 llamafactory-cli这个文件，也即是必须后面需要跟train.py文件？

command:

/bin/bash
-c
"dlrover-run --network-check --nnodes=$NODE_NUM \ --nproc_per_node=1 --max_restarts=1 \ llamafactory-cli train examples/train_lora/llama3_lora_sft.yaml"