请教一下训练速度问题

s-JoL / Open-Llama

The complete training code of the open-source high-performance Llama model, including the full process from pre-training to RLHF.

https://huggingface.co/s-JoL/Open-Llama-V2

MIT License

30 stars 4 forks source link

Closed ray075hl closed 2 months ago

ray075hl commented 3 months ago

您好我看llama1里面7b的训练速度3300tokens/gpu/second 是在2000多张卡的集群上训练的速度；如果我只有5台机器40卡，是不是应该速度应该比它快很多啊？