search

project-baize / baize-chatbot

Let ChatGPT teach your own chatbot in hours with a single GPU!
https://arxiv.org/abs/2304.01196
GNU General Public License v3.0
3.15k stars 277 forks source link

get bug when use finetune code #7

Closed AItechnology closed 1 year ago

AItechnology commented 1 year ago

When I execute finetune code, it gets such a bug RuntimeError: module must have its parameters and buffers on device cuda:0 (device_ids[0]) but found one of them on device: cuda:1

JetRunner commented 1 year ago

$ export CUDA_VISIBLE_DEVICES=0 will fix that. Feel free to implement a data parallel pipeline if you'd like to use multiple GPUs to train the model.

AItechnology commented 1 year ago

Thanks very much, I can finetune the model, I just want to use multi GPUs for finetuning a large model, I had tried some methods but failed, so will you provide a multi-GPU finetune code for the data parallel pipeline?