search

shibing624 / textgen

TextGen: Implementation of Text Generation models, include LLaMA, BLOOM, GPT2, BART, T5, SongNet and so on. 文本生成模型，实现了包括LLaMA，ChatGLM，BLOOM，GPT2，Seq2Seq，BART，T5，UDA等模型的训练和预测，开箱即用。
Apache License 2.0
938 stars 109 forks source link

hello master，BLOOM 模型 目前支持单机多卡训练么，如果支持是 #42

Open 116705792 opened 1 year ago

116705792 commented 1 year ago

Describe the Question

Please provide a clear and concise description of what the question is.

Describe your attempts

116705792 commented 1 year ago

如果支持的话，也是这种形式么？ CUDA_VISIBLE_DEVICES=0,1 torchrun --nproc_per_node 2 training_chatglm_demo.py --do_train --do_predict --num_epochs 20

shibing624 commented 1 year ago

是

stale[bot] commented 11 months ago

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.(由于长期不活动，机器人自动关闭此问题，如果需要欢迎提问)