search

HarderThenHarder / transformers_tasks

⭐️ NLP Algorithms with transformers lib. Supporting Text-Classification, Text-Generation, Information-Extraction, Text-Matching, RLHF, SFT etc.
https://www.zhihu.com/column/c_1451236880973426688
2.16k stars 381 forks source link

关于UIE模型微调，如何进行一机多卡训练？ #47

Open zhougz2018 opened 1 year ago

HarderThenHarder commented 1 year ago

Hi，您可以在 这一行后面 通过加入下面一行简单的命令来实现多卡训练：

...
model.to(args.device)

[+] model = torch.nn.DataParallel(model, device_ids=[0, 1, 2])            # 需要用的显卡索引

tokenizer = AutoTokenizer.from_pretrained(args.pretrained_model)
...
zhougz2018 commented 1 year ago

我的文本长度 5000 ，内存就爆了，数据需要处理吗？