search

linkedin / Liger-Kernel

Efficient Triton Kernels for LLM Training
BSD 2-Clause "Simplified" License
2.93k stars 147 forks source link

[feat] Is Qwen1.5 supported? #133

Closed YingchaoX closed 3 weeks ago

YingchaoX commented 3 weeks ago

🚀 The feature, motivation and pitch

I am currently using qwen1.5-14b, and I'm not sure if the templates for qwen2 are compatible with qwen1.5.

Alternatives

No response

Additional context

No response

ByronHsu commented 3 weeks ago

cc @chiwanpark

chiwanpark commented 3 weeks ago

@YingchaoX cc @ByronHsu AFAIK, You can apply Liger patches to Qwen1.5 because Qwen1.5 shares the architecture with Qwen2 (config). But, I didn't test it yet.

ByronHsu commented 3 weeks ago

i believe it should work

YingchaoX commented 2 weeks ago

great, it works and much faster than before, thanks