adrianeboyd
commented
1 year ago Let me first back up a step: your goal is to have a final pipeline with everything from en_core_web_trf + your new textcat component (non-transformer-based) trained on your data from prodigy?
If that's the case, then you can train the textcat model separately and "assemble" the final pipeline as the last step. It would look like this:
prodigy data-to-spacy out/ --textcat-multilabel dataset_name --eval-split 0.2
spacy train out/config.cfg --paths.train out/train.spacy --paths.dev out/dev.spacy -o training/And then assemble. You can write a config to do with spacy assemble, but it's easier to do it programmatically:
import spacy
nlp = spacy.load("en_core_web_trf")
tcm_nlp = spacy.load("training/model-best")
tcm_nlp.replace_listeners("tok2vec", "textcat_multilabel", ["model.tok2vec"])
nlp.add_pipe("textcat_multilabel", source=tcm_nlp)
nlp.to_disk("/path/to/my_combined_pipeline")In addition, the prodigy config defaults that you get with data-to-spacy are for the faster BOW-only textcat architecture, when you may see better performance with the ensemble classifier. The config could look like this:
[components.textcat_multilabel]
factory = "textcat_multilabel"
scorer = {"@scorers":"spacy.textcat_multilabel_scorer.v1"}
threshold = 0.5
[components.textcat_multilabel.model]
@architectures = "spacy.TextCatEnsemble.v2"
nO = null
[components.textcat_multilabel.model.linear_model]
@architectures = "spacy.TextCatBOW.v2"
exclusive_classes = false
ngram_size = 1
no_output_layer = false
nO = null
[components.textcat_multilabel.model.tok2vec]
@architectures = "spacy.Tok2VecListener.v1"
width = ${components.tok2vec.model.encode.width}
upstream = "*"
[components.tok2vec]
factory = "tok2vec"
[components.tok2vec.model]
@architectures = "spacy.Tok2Vec.v2"
[components.tok2vec.model.embed]
@architectures = "spacy.MultiHashEmbed.v2"
width = ${components.tok2vec.model.encode.width}
attrs = ["NORM","PREFIX","SUFFIX","SHAPE"]
rows = [5000,1000,2500,2500]
include_static_vectors = false
[components.tok2vec.model.encode]
@architectures = "spacy.MaxoutWindowEncoder.v2"
width = 256
depth = 8
window_size = 1
maxout_pieces = 3(Ideally you'd be able to generate this with spacy init config -p textcat_multilabel, but the default options only allow for BOW (-o efficiency) or ensemble+static vectors (-o accuracy). The version above is modified by hand from -o accuracy to disable static vectors, which you don't have in the en_core_web_trf pipeline. If you used en_core_web_lg instead, you could keep the static vectors enabled.)
cbjrobertson
I'm training a
spaCymulti-label text classification model usingen_core_web_trffromspaCytransformers. The data and config file I use are generated through aprodigy data-to-spacycall. The issue is that when I try to reload the model usingspacy.load("path/to/mod"), it returns:ValueError: Cannot deserialize model: mismatched structure. Based on theprodigyforums, this ought to have been fixed withspacy-transformersversion1.0.6but I'm runningspacy-transformers==1.1.8so I believe there's still a bug somewhere, likely inspacy-transformers, see #8566. I can't share the data, but I'll do my best to make the issue reproducible.How to reproduce the behaviour (from a notebook):
On the basis of this advice, this error can be worked around in either of two ways:
spacy.load(out_path)tospacy.load(out_path,disable="tagger,parser,attribute_ruler,lemmatizer,ner")out_path/config.cfgfrompipeline = ["transformer","tagger","parser","attribute_ruler","lemmatizer","ner","textcat_multilabel"]topipeline = ["transformer","textcat_multilabel"]However, neither of these are ideal. For instance, getting these models to work with spacy-report requires editing the source code of that package. It seems there's still a bug relating to the frozen components in the call to
spacy.train!Here is an example observation from
dataset_name:Your Environment
spacy validate: