search

stefan-it / turkish-bert

Turkish BERT/DistilBERT, ELECTRA and ConvBERT models
494 stars 42 forks source link

Using turkish bert with tensorflow or tf.keras #7

Open cansozbir opened 4 years ago

cansozbir commented 4 years ago

I want to combine this model with a CNN in tensorflow or tensorflow.keras but couldn't handle how to use these checkpoint files. Can someone help me about how to use this model in tensorflow.keras?

file contains : config.json model.ckpt.data-00000-of-00001 model.ckpt.index model.ckpt.meta vocab.txt

@balki7 Sure, here are the original TF checkpoints (best checkpoint, incl. config and vocab):

wget https://schweter.eu/cloud/bert-base-turkish-cased/bert-base-turkish-cased-tf.tar.gz

# sha256 bert-base-turkish-cased-tf.tar.gz
# 8113d0aeb32a2e7bcd00027195a13622387ca9e2132d7f9a1b27389a0db26b96

Thank u so much Stefan.

Originally posted by @balki7 in https://github.com/stefan-it/turkish-bert/issues/2#issuecomment-589930999

stefan-it commented 4 years ago

Hi @cansozbir ,

you can now use the TF-compatible weights for all Turkish BERT models:

from transformers import  TFBertModel  

model = TFBertModel.from_pretrained('dbmdz/bert-base-turkish-cased')

I haven't worked with Transformers in combination with Keras yet, but I think this issue could be helpful to get it working:

https://github.com/huggingface/transformers/issues/1350

mdeniz1 commented 1 year ago

Hi @stefan-it I have used it with keras and it works fine but in a more recent issue from 2021 you again refer to downloading the weights instead of using the usual checkpoint. So which one should we use? Should we download it or use the checkpoint above? Mehmet