PetrochukM / PyTorch-NLP

Basic Utilities for PyTorch Natural Language Processing (NLP)
https://pytorchnlp.readthedocs.io
BSD 3-Clause "New" or "Revised" License
2.21k stars 257 forks source link

SpacyWordSplitter: module not found - AllenNLP v1.5.0 #125

Closed vrunm closed 1 year ago

vrunm commented 1 year ago

I have been using AllenNLP v1.5.0 for migrating old code. What is the version of spacy that works well AllenNLP v1.5.0 ? I have tried using spacy 2.1.0 but still ran into some import issues.

Also I got an import error as :

from allennlp.data.tokenizers.word_splitter import SpacyWordSplitter
ModuleNotFoundError: No module named 'allennlp.data.tokenizers.word_splitter'

Also allennlp.data.tokenizers.word_splitter no longer exists in the latest version of AllenNLP. It has been replaced with SpacyTokenizer. So what would be the correct version of Spacy to use for a tokenizer in AllenNLP v1.5.0?