Get rid of FutureWarning

cisnlp / simalign

Obtain Word Alignments using Pretrained Language Models (e.g., mBERT)

MIT License

345 stars 47 forks source link

Get rid of FutureWarning #19

Closed xz-liu closed 3 years ago

xz-liu commented 3 years ago

is_pretokenized is deprecated and will be removed in a future version, use is_split_into_words instead."

pdufter commented 3 years ago

Thanks @joker-xii

I will revert back to "is_pretokenized", as "is_split_into_words" is not supported by v.3.1.0 (which is the version we currently use). Sorry for the confusion, mixed up transformer versions...