eubinecto / idiomatch

An implementation of SpaCy(3.0)'s Matcher specifically designed for identifying English idioms.
40 stars 6 forks source link

shipping the model #5

Closed eubinecto closed 3 years ago

eubinecto commented 3 years ago

Why?

In order to use the pipeline outside of this library. (e.g. idiom2vec & youtora)

The goal

distribute mip on a package.. so that I could do.. pip install mip_0.0.0?

But how?

there have been attempts to ship a model with a serialisable tokenizer.

but obviously. my idiom_matcher is not serialisable. what a shame.

shipping with custom component?

you can ship the entire model package!

To-do's

eubinecto commented 3 years ago

okay..well. It seems it's not just plug and play.

You have to install the library merge_idioms, before you install the model. hmm.. this is bad.. I want a single package which I can just ship.

what about this one?

This person had exactly the same question that I had here.

By default, the init.py only includes a function to load your model – but you can also add custom functions to it or use it to add entries to spaCy's factories.

and..the problem with loading matcher from idiom_matcher.pkl

What if someone else want's to attach the component to their model? The current way of loading pkl and syncing the vocab will mess up the previous vocab.

eubinecto commented 3 years ago

a very good blog post on uploading the project on pip

eubinecto commented 3 years ago

to distribute your library to pypi, just run:

twine dist/*