Do the reverse embedding in the same dtype as the input embedding

keras-team / keras-nlp

Modular Natural Language Processing workflows with Keras

Apache License 2.0

740 stars 218 forks source link

Closed mattdangerw closed 3 months ago

mattdangerw commented 3 months ago

Fixes #1542

mattdangerw commented 3 months ago

Looks good, thanks for the fix! I haven't tested with Gemma but LLaMA and Mistral work.

Thanks! I am testing with Gemma now.

mattdangerw commented 3 months ago

I think this looks good to go, but to be safe I will probably wait till after cloud next to push our later next week.