keras-team / keras-nlp

Modular Natural Language Processing workflows with Keras
Apache License 2.0
734 stars 216 forks source link

Any plans for more Llama type models? #1587

Open pass-lin opened 2 months ago

pass-lin commented 2 months ago

Is there a plan to support more models similar to llama, which only need simple modifications or even no modifications to the existing llama backbone to achieve compatibility?

for example yi model :https://huggingface.co/01-ai This model is exactly the same as llama.
qwen model :https://huggingface.co/Qwen This model only need to modify three attention bias

Are there plans to support even larger llama models, such as the llama 70B model?

tirthasheshpatel commented 2 months ago

I plan to add larger LLaMA presets in KerasNLP. No plans for Yi or Quen yet.