search

keras-team / keras-hub

Pretrained model hub for Keras 3
Apache License 2.0
804 stars 243 forks source link

Simplify registering "built-in" presets #1818

Closed mattdangerw closed 2 months ago

mattdangerw commented 2 months ago

Instead of registering classes with backbone and tokenizer, we just register them with the associated backbone. This will keep the accessors working for models without tokenizer classes. We can build the correct list for any class in the library by checking the associated backbone class.

keras_nlp.models.PaliGemmaTokenizer.presets.keys()
keras_nlp.models.Gpt2Backbone.presets.keys()
keras_nlp.models.TextClassifier.presets.keys()
keras_nlp.layers.AudioConverter.presets.keys()
SamanehSaadat commented 2 months ago

Instead of registering classes with backbone and tokenizer, we just register them with the associated backbone.

What was the reason for registering the tokenizer before this change?

mattdangerw commented 2 months ago

What was the reason for registering the tokenizer before this change?

Previously we didn't have the tokenizers associated with a backbone class. But not a particularly good reason, just hadn't thought of this approach?