refactors, standardizes, and expands document similarity metrics
top-level module is now a subpackage, with metrics split out into categories: edit-based, token-based, etc.
adds several similarity metrics: edit-based Jaro; token-based Cosine, Bag, and Tversky; sequence-based Matching Subsequences Ratio; and hybrid Monge-Elkan
removes a couple similarity metrics: Word Movers Distance relied on a troublesome external dependency, and Word2Vec+Cosine is directly available in spaCy via Doc.similarity (and likewise for Span and Token)
leverages spaCy's new DocBin functionality for efficiently saving/loading Docs in binary format
custom hacks behind io.read_spacy_docs() and io.write_spacy_docs() funcs have been replaced with official spaCy usage, and function args have been updated to match
removes textacy.export module, which had a couple functions for exporting spaCy docs into gensim- and CONLLU-friendly formats; the use case seemed minimal, and it went against efforts to refine and constrain connections with external packages
updates package dependencies list and their versions
removes: pyemd and srsly
un-caps: numpy and scikit-learn
bumps: cytoolz, jellyfish, and pyphen
reorganizes and tidies up some module docs and API reference docs
expands use of package-level types, and tidies up many type annotations
Motivation and Context
Another chunk of work aimed at sweeping up textacy's many cobwebs, better and more smartly leveraging spaCy v3's functionality, and refining our connections with other external packages.
How Has This Been Tested?
many tests improved, and all tests pass
Screenshots (if appropriate):
Types of changes
[x] Bug fix (non-breaking change which fixes an issue)
[x] New feature (non-breaking change which adds functionality)
[x] Breaking change (fix or feature that would cause existing functionality to change)
Checklist:
[x] My code follows the code style of this project.
[x] My change requires a change to the documentation, and I have updated it accordingly.
Description
Doc.similarity(and likewise for Span and Token)DocBinfunctionality for efficiently saving/loadingDocs in binary formatio.read_spacy_docs()andio.write_spacy_docs()funcs have been replaced with official spaCy usage, and function args have been updated to matchtextacy.exportmodule, which had a couple functions for exporting spaCy docs intogensim- and CONLLU-friendly formats; the use case seemed minimal, and it went against efforts to refine and constrain connections with external packagespyemdandsrslynumpyandscikit-learncytoolz,jellyfish, andpyphentypes, and tidies up many type annotationsMotivation and Context
Another chunk of work aimed at sweeping up
textacy's many cobwebs, better and more smartly leveraging spaCy v3's functionality, and refining our connections with other external packages.How Has This Been Tested?
many tests improved, and all tests pass
Screenshots (if appropriate):
Types of changes
Checklist: