search

jzhang38 / TinyLlama

The TinyLlama project is an open endeavor to pretrain a 1.1B Llama model on 3 trillion tokens.
Apache License 2.0
7.7k stars 453 forks source link

lightning run model #191

Open icemoon-creative opened 3 months ago

icemoon-creative commented 3 months ago

lightning run model \ --node-rank=0 \ --main-address=172.16.101.5 \ --accelerator=cuda \ --devices=8 \ --num-nodes=2 \ pretrain/tinyllama.py --devices 8 --train_data_dir data/slim_star --val_data_dir data/slim_star 您好，我想请问一下lightning run model是如何启动服务的，并没有看到pytorch lightning有这样的命令啊

ChaosCodes commented 3 months ago

你可以试试 fabric run，lightning 最新版好像不支持 lightning run 了