Open Mrutyunjay01 opened 1 month ago
added subclassing implementation. Certain obstacles remain as mentioned:
BertEncoderLayer
class, as it is inherited from layers.Layer
, is it possible to access to the sub-layers here? as:
mlm_model = models.load_model("bert_mlm.keras", custom_objects={"MaskedLanguageModel": MaskedLanguageModel})
mlm_model.get_layer("bert_encoder_layer").get_layer("encoder_0_ffn_layernormalization")
This throws error saying
```py
AttributeError: 'BertEncoderLayer' object has no attribute 'get_layer'
# Load pretrained bert model
mlm_model = models.load_model(
"bert_mlm_imdb.keras", custom_objects={"MaskedLanguageBertModel": MaskedLanguageBertModel}
)
pretrained_bert_model = Model(
mlm_model.input, mlm_model.get_layer("encoder_0_ffn_layernormalization").output
)
This was possible in functional declaration of the Model, but not in sub-classing (in last commit).
Kindly suggest w.r.t last commit
Thanks!
With reference to the draft PR, here's an attempt to make to example truly backend-agnostic. In the process, I raised the following issues: https://github.com/keras-team/keras/issues/18410, and https://github.com/keras-team/keras/issues/19665. While the latter got resolved, the former persists still. Kindly review the changes, and let me know how shall we proceed with the issues mentioned.
cc: @fchollet