search

kpe / bert-for-tf2

A Keras TensorFlow 2.0 implementation of BERT, ALBERT and adapter-BERT.
https://github.com/kpe/bert-for-tf2
MIT License
802 stars 193 forks source link

will albert for tf2 be released? #7

Closed breadbread1984 closed 4 years ago

breadbread1984 commented 4 years ago

will albert for tf2 be released?

kpe commented 4 years ago

I have not yet evaluate exactly how much effort it would be to load some of the released albert weights, but I'm planning to do so soon. Until recently there were no pre-trained models for English, but I believe they might be released soon (according to https://github.com/brightmart/albert_zh?source=post_page-----df8f7b58fa28----------------------#%E5%8F%91%E5%B8%83%E8%AE%A1%E5%88%92-release-plan), so it would be worth evaluating.

breadbread1984 commented 4 years ago

please, thx

kpe commented 4 years ago

I added some initial support for the ALBERT model through the shared_layer=True and the embedding_size parameters. However no support for loading of the released pre-trained weights yet.

breadbread1984 commented 4 years ago

thx for your speedy support of the new model!

kpe commented 4 years ago

with the latest commit bert-for-tf2 supports now loading of the pre-trained ALBERT weights released by @brightmart. I very much hope there we'll be pre-trained weights for English too soon, see also brightmart/albert_zh#11

brightmart commented 4 years ago

great job!
@kpe @breadbread1984

breadbread1984 commented 4 years ago

@kpe you are great! @brightmart I am sharing the honor by mistake.