Add default LoRA target modules for Phi-2

ludwig-ai / ludwig

Low-code framework for building custom LLMs, neural networks, and other AI models

http://ludwig.ai

Apache License 2.0

11.21k stars 1.19k forks source link

Closed arnavgarg1 closed 10 months ago

arnavgarg1 commented 10 months ago

These modules maximize performance, also the current LoRA target modules for Phi-2 are wrong on the PEFT repo.

See #3910

github-actions[bot] commented 10 months ago

  6 files ±0   6 suites ±0 14m 15s :stopwatch: -11s 12 tests ±0   9 :heavy_check_mark: ±0   3 :zzz: ±0 0 :x: ±0 60 runs ±0 42 :heavy_check_mark: ±0 18 :zzz: ±0 0 :x: ±0

Results for commit a1baa71c. ± Comparison against base commit 91c8975f.