keras-team / keras-nlp

Modular Natural Language Processing workflows with Keras
Apache License 2.0
758 stars 227 forks source link

Add LLaMA Causal LM with 7B presets #1526

Closed tirthasheshpatel closed 5 months ago

tirthasheshpatel commented 5 months ago

This PR adds the LLaMA Causal LM along with a weight conversion script for the 7B presets (LLaMA 7B and LLaMA Chat 7B).

Tested that the outputs of the models match with 1e-4 tolerance on CPU and float32.

TODO:

tirthasheshpatel commented 5 months ago

Changes on master are breaking the tests in this PR. Will sync and push the final changes.

mattdangerw commented 5 months ago

Thanks!