search

utterworks / fast-bert

Super easy library for BERT based NLP models
Apache License 2.0
1.85k stars 342 forks source link

ImportError: cannot import name 'BertWordPieceTokenizer' #174

Open stellaywu opened 4 years ago

stellaywu commented 4 years ago

tried installed from pip and source, both returned ImportError: cannot import name 'BertWordPieceTokenizer' on from fast_bert.data_cls

Please help!

mrjgamble commented 4 years ago

I have the same issue. I've tried using 1.6.1 and 1.6.0, but then run into other problems during the fit of a multi-label model.

dhruv-sharma commented 4 years ago

This issue may be due to the version of the tokenizers module. If so then you would need to first install Rust and then install the tokenizers module.

Assuming you have a bash shell and are within your env (actual or venv development environment), you can try the following:

curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh source $HOME/.cargo/env pip install tokenizers

More logs would help though.

propower1712 commented 4 years ago

I solved the issue by adding those two lines befort the import :

import torch

camembert = torch.hub.load('pytorch/fairseq', 'camembert.v0')

Shane-Neeley commented 4 years ago

I had this and then ran pip install --upgrade tokenizers and got past it. Though I'm still nowhere near getting it to work yet ha