Huggingface support for Tensorflow?

[jacklxc]

It seems that SciBERT only supports Huggingface with PyTorch backend only, but not Tensorflow backend. Can you also provide the SciBERT support for Huggingface Tensorflow backend, or is there any workaround for this purpose? Thank you!

[ibeltagy]

I am not familiar with how HF TF support works, but as far as I understand, we don't need to do anything specific on the model side to make it work on one or the other. I would expect something like TFBertModel.from_pretrained('allenai/scibert_scivocab_uncased') to work out of the box

[jacklxc]

Hi @ibeltagy, as far as I have tried, TFBertModel.from_pretrained('allenai/scibert_scivocab_uncased') or TFAutoModel.from_pretrained('allenai/scibert_scivocab_uncased') throws an error saying the path is not correct. However the similar operation for PyTorch introduced here works without error.

[vasudev13]

Hi, when working with huggingface/transformers as under

from transformers import AutoTokenizer, AutoModel
tokenizer = AutoTokenizer.from_pretrained("allenai/scibert_scivocab_uncased")
model = AutoModel.from_pretrained("allenai/scibert_scivocab_uncased")

the following error occurs:

OSError: Can't load config for 'allenai/scibert_scivocab_uncased'. Make sure that:

- 'allenai/scibert_scivocab_uncased' is a correct model identifier listed on 'https://huggingface.co/models'

- or 'allenai/scibert_scivocab_uncased' is the correct path to a directory containing a config.json file

Similar error occurs when using TFBertModel as well. @ibeltagy would be great if you can guide on how to work around the above

Thanks a lot.

[dalgaard]

Hi,

I am facing the same issues as reported by @jacklxc, @ibeltagy do you have any update on this ?

Thanks

[dalgaard]

Hi,

I am facing the same issues as reported by @jacklxc, @ibeltagy do you have any update on this ?

Thanks

Hi again,

I got it working just adding the parameter from_pt=True to the method call fixes my issue:

from transformers import AutoTokenizer, AutoModel
tokenizer = AutoTokenizer.from_pretrained("allenai/scibert_scivocab_uncased", from_pt=True)
model = AutoModel.from_pretrained("allenai/scibert_scivocab_uncased", from_pt=True)

[jacklxc]

@dalgaard @vasudev13 It seems your discussion is drifting to the PyTorch version of SciBERT. @dalgaard Have you tried for TFBertModel?

[dalgaard]

@jacklxc yes you are right, sorry. I actually only had the problem for the TF version, PyTorch seemed to always work for me. So the code in my last post should have used TFAutoModel or TFBertModel, both classes work for me with the parameter from_pt=True.

[vasudev13]

Yes, same for me.

Thanks.