Closed iliaschalkidis closed 5 years ago
Hi @iliaschalkidis,
you can use train_model
with from transformer.train import train_model
and define a non token-level Task
and train the model on your sentence-level task(which uses [CLS]
internally)
you can take a look at tutorial.ipynb and read this in code.
Hi @Separius, this looks really promising. Although I want to build something more complicated (custom), so I am going to read your code for train_model
and start prototyping my model based on these principles.
I wish Google had already port BERT into tensorhub and made it as easy to use as ELMo tensorhub module.
Thanks a lot for your quick response! Have a great week!
Is there any possible way to train a BERT-based classifier, using [CLS] vector, as described in BERT article?
I was able to load BERT encoder successfully using:
, but I am not able to find a workaround in order to wrap the encoder in a keras Model (wrapper).
I was looking forward to something like that:
If there is such a solution, it would be the easiest way possible to use BERT!