search

AnswerDotAI / fsdp_qlora

Training LLMs with QLoRA + FSDP
Apache License 2.0
1.42k stars 188 forks source link

train.py #63

Open mylesgoose opened 7 months ago

mylesgoose commented 7 months ago

I had to vary this code here in the Train.py to get it to work on my system

LoRA and DORA modules

sys.path.append("./scripts") from scripts.lora import LORA from scripts.dora import BNBDORA, HQQDORA, DORALayer, MagnitudeLayer

probably because i already had a module called dora and lora in the pip.

MizuleGPT commented 5 months ago

this fixed the problem for me too, you probably saved me a week of mindless scanning and screaming at AIs