Closed iamwangyabin closed 1 year ago
我看好像get_param_group_train逻辑里面把lora训练包含进去了,但是参数似乎还可以一起优化
Currently, I'm sure that fine-tuning with a single card has no problem in my environment. However, this bug still occurs no matter how I try to fix the code. Like this, setting find_unused_parameters=True is useless.
This bug is fix in the latest version. Fxck the DDP!
Config files I use:
Single card training and multi-gpu training with LoRA work fine.