search

kyzhouhzau / NLPGNN

1. Use BERT, ALBERT and GPT2 as tensorflow2.0's layer. 2. Implement GCN, GAN, GIN and GraphSAGE based on message passing.
MIT License
331 stars 65 forks source link

using another pre-trained BERT model #1

Open GhadaAlfattni opened 4 years ago

GhadaAlfattni commented 4 years ago

Hi

I would like to train a NER (EN) but with my own BERT model, where can I do this.

Thanks

GhadaAlfattni commented 4 years ago

and what about the data, can I use something like this

sulfate 100130 547 554 B-Chem S-Chem

Or the data must be exactly similar to conll?

kyzhouhzau commented 4 years ago

@GhadaAlfattni

If you want to use your own model, first you must make sure that the parameter format of the model is the same as that provided by BERT official, otherwise you will encounter some problems. In fact, I provided a url parameter in the LoadCheckpoint class to pass the model you have trained. This parameter needs to be a link, and the data is the same compression format as the official BERT file. example: "https://storage.googleapis.com/bert_models/2018_11_03/chinese_L-12_H-768_A-12.zip" or :"https://XXXXX/XXXXX/XXXXX.zip"

In order to facilitate the implementation of multiple tasks based on BERT under the framework, I have changed the input format to "sentence \t sequence_labels" format, which does not support the format you provide.

But if you want to use a custom format, you can customize the data load method without changing the model architecture.

GhadaAlfattni commented 4 years ago

I'll check then

Thanks @kyzhouhzau