search

h2oai / h2ogpt

Private chat with local GPT with document, images, video, etc. 100% private, Apache 2.0. Supports oLLaMa, Mixtral, llama.cpp, and more. Demo: https://gpt.h2o.ai/ https://gpt-docs.h2o.ai/
http://h2o.ai
Apache License 2.0
11.44k stars 1.25k forks source link

fsdp error #671

Open charisliu123 opened 1 year ago

charisliu123 commented 1 year ago

i run the finetune.py without lora and use the fsdp ,but there is a error, fsdp/_init_utils.py,line889 inconsistent compute device and'device_id ' on rank 3: cuda:0 vs cuda:3; i run the programmer in Single machine multi card

arnocandel commented 1 year ago

Yes, WIP: https://github.com/h2oai/h2ogpt/pull/648/, known regression from the early days.