search

iPieter / RobBERT

A Dutch RoBERTa-based language model
https://pieter.ai/robbert/
MIT License
196 stars 29 forks source link

Using the model in tensorflow #3

Closed melvinvandenbout closed 4 years ago

melvinvandenbout commented 4 years ago

Would it be possible to use this model in tensorflow? I think i need the pre-trained model in a format like (bert_model.ckpt, vocab.txt, bert_config.json), just like https://github.com/google-research/bert#pre-trained-models.

twinters commented 4 years ago

Thanks for your interest! I personally don't see an easy way of achieving this. I heard there are ways of translating PyTorch models to Tensorflow models using ONNX, but RobBERT is probably too complex for that. You might want to search if someone ever succeeded in translating RoBERTa to Tensorflow: if you have such an algorithm, it would trivially work for our RobBERT model as well.

I think these kind of incompatibilities is probably why a lot of people (including @iPieter and me) shift towards using HuggingFace transformers, as they provide a nice overarching framework for these kind of models. Might we worthwhile changing your set-up to their framework if you want to use multiple different types of transformers.

Hope this helps!

melvinvandenbout commented 4 years ago

Thanks for your elaboration & insightful comment!