The new version of spaCy can reuse an existing tokenization layer. This greatly simplifies alignment so the code should be changed for that. Unfortunately it is not a good idea to throw out the old complicated code since spaCY can still run in standalone mode.
The new version of spaCy can reuse an existing tokenization layer. This greatly simplifies alignment so the code should be changed for that. Unfortunately it is not a good idea to throw out the old complicated code since spaCY can still run in standalone mode.