Example for training a BERT model from scratch using new data without using pre-trained weights

kpe / bert-for-tf2

A Keras TensorFlow 2.0 implementation of BERT, ALBERT and adapter-BERT.

https://github.com/kpe/bert-for-tf2

MIT License

803 stars 193 forks source link

Example for training a BERT model from scratch using new data without using pre-trained weights #51

Open wenbostar opened 4 years ago

wenbostar commented 4 years ago

Hi,

The library looks great. I'm trying to use BERT for biological sequence tasks so I cannot use the pre-trained models with language data. Could you provide an example to show how to train a BERT model from scratch using new data without using any pre-trained weights?

Thank you in advance.

antopemo commented 4 years ago

I would love to see this too, I need to pretrain for a different task and an example would be great!