search

lapp0 / distily

Distily: Language Model Distillation Toolkit and Library
GNU Affero General Public License v3.0
2 stars 0 forks source link

Integrate Liger #2

Open lapp0 opened 4 weeks ago

lapp0 commented 4 weeks ago

https://github.com/linkedin/Liger-Kernel

Should be easy with https://github.com/linkedin/Liger-Kernel/issues/242#issuecomment-2347334135

Also we should patch kl_div loss with ligers kl_div if liger is available.

ByronHsu commented 3 weeks ago

thanks :-)