Closed borisdayma closed 5 months ago
Thanks for the report! Will add the new presets.
Thanks a lot! Btw only the instruct variant has a version 0.2 but it's quite improved from 0.1
Thanks for this info. #1520 adds the preset. It should be available on Kaggle in around an hour and will be accessible in the next release of KerasNLP.
Is your feature request related to a problem? Please describe.
We can currently load Mistral 7b models with
keras_nlp.models.MistralCausalLM.from_preset("mistral_7b_en")
(ormistral_instruct_7b_en
). I noticed those are the version0.1
of the models. The versions0.2
have significantly improved while using the same code base.Describe the solution you'd like
It would be nice to offer the
0.2
variants of both base and instruct models as possible presets.Describe alternatives you've considered
Just using the
0.1
versions or other libraries… However Keras offers nice integration with JAX + sharding!Additional context
I want to use Keras models as backbones to train VLM’s and Mistral is a very strong variant at 7B.