search

Lightning-AI / litgpt

Pretrain, finetune, deploy 20+ LLMs on your own data. Uses state-of-the-art techniques: flash attention, FSDP, 4-bit, LoRA, and more.
https://lightning.ai
Apache License 2.0
6.85k stars 726 forks source link

Update LoRA test #1376

Closed awaelchli closed 2 weeks ago

awaelchli commented 2 weeks ago

1374 changed the assertion but it's not needed. The values are exactly equal. Also the function used there is deprecated in PyTorch.