free-variation
commented
5 years ago We should adopt in some form of the way they set up pipelines, which mimic some of the Spacy API behavior.
Agreed. This is what @kylepjohnson sketched out: a spaCy-like nlp callable object that functions as the entry point to the pipeline. (Since we're in a multilingual context, we might suggest a convention where the name of the object is that of the language: latin, oe, etc.)
we should not make StanfordNLP a direct dependency. We should create wrappers which may load and interact with their published Pytorch models,
I want to make sure I understand this. Are you proposing that we not make stanfordnlp the only library for building and running statistical models, and instead push the dependencies to the modules responsible for loading Stanford models for specific languages?
While CoNLL is laudable, the languages under CLTK's purview typically have limited resources in the way of available corpora, and standardized annotations. In our desire and long term goal to achieve state of the art, it's important that leave the door open to custom training data sets that won't be restricted to the CoNLL format.
I think I basically agree, but what if the problem is just one of writing converters from treebank X to CoNLL? I face this problem directly with OE. I've trained a parser, using the small ISWOC treebank. Accuracy isn't great (though not awful), so I'd like to use the much larger YCOE. Unfortunately the latter uses its own idiosyncratic annotations, and does not provide lemmas, which are required by the stanfordnlp trainer. POS tags too.
What to do? The more tractable path I'd think would be to convert the old YCOE treebank to CoNLL, run the corpus through a lemmatizer and POS tagger, and then train using the Stanford toolset. I suppose one could also try to unpeel the CoNLL bits, to feed the PyTorch models directly. More laborious still would be to rewrite the model structure of the Stanford NN in our own PyTorch (or even Keras!) representations. On balance though I'd probably start with translating the files and producing the additional, required representations -- an issue being that errors by the lemmatizer and POS tagger will end up contaminating the training file.
Let's see, what's my point :) ? Oh, just that it may be more practical to always (usually?) convert treebanks to CoNLL, in which case the dependency problem isn't particularly severe. I nevertheless agree that it smells better to not build in a hard dependency.
kylepjohnson
todd-cook
Elsewhere use and possible inclusion of StanfordNLP Python library (https://github.com/stanfordnlp/stanfordnlp) has been discussed, e.g. here: (https://github.com/cltk/cltk/issues/921)
I've examined this library and would like to share my recommendations, reasons, and hunches.
Recommendations:
We should adopt in some form of the way they set up pipelines, which mimic some of the Spacy API behavior.
However we should not make StanfordNLP a direct dependency. We should create wrappers which may load and interact with their published Pytorch models, which we may allow users to download and install. Pytorch itself may reasonably become a first class CLTK dependency.
Reasons: The basic approach of StanfordNLP Python is to generate general purpose models based on the standardized CoNLL input data format. This has allowed StanfordNLP to leverage a standard format, and encourage repeatable, independently verifiable model generation, across a wide number of languages.
While CoNLL is laudable, the languages under CLTK's purview typically have limited resources in the way of available corpora, and standardized annotations. In our desire and long term goal to achieve state of the art, it's important that leave the door open to custom training data sets that won't be restricted to the CoNLL format.