Add default LoRA target modules for Phi-2

ludwig-ai / ludwig

Low-code framework for building custom LLMs, neural networks, and other AI models

http://ludwig.ai

Apache License 2.0

11.11k stars 1.19k forks source link

Closed arnavgarg1 closed 8 months ago

arnavgarg1 commented 8 months ago

These modules maximize performance, also the current LoRA target modules for Phi-2 are wrong on the PEFT repo.

See #3910

github-actions[bot] commented 8 months ago

  6 files ±0   6 suites ±0 14m 15s :stopwatch: -11s 12 tests ±0   9 :heavy_check_mark: ±0   3 :zzz: ±0 0 :x: ±0 60 runs ±0 42 :heavy_check_mark: ±0 18 :zzz: ±0 0 :x: ±0

Results for commit a1baa71c. ± Comparison against base commit 91c8975f.