snorkel-team / snorkel

A system for quickly generating training data with weak supervision
https://snorkel.org
Apache License 2.0
5.81k stars 857 forks source link

how to export model trained on snorkel? #861

Closed arturomp closed 6 years ago

arturomp commented 6 years ago

How do I use the model I trained on Snorkel on a non-jupyter/non-snorkel environment?

For instance, the model would be the end relation extraction model at the end of Part III of the intro tutorial. I've been looking around and stumbled into Q&A #726 to use the trained model within snorkel, but I don't see how the model can be exported.

My question is similar to this stackoverflow question on exporting a trained model on sklearn.

ajratner commented 6 years ago

Hi @arturomp ,

With respect to running Snorkel outside of the Jupyter notebooks- this should be straightforward! Just import Snorkel as you would any other Python package. Anything other than the visualization stuff (i.g. the Viewer) will work just fine!

As for using a model trained in Snorkel without utilizing any other Snorkel utilities: this is conceptually very simple, but would require some new glue code. Currently, when a (Tensorflow) model is trained in Snorkel, the model checkpoint (which is Tensorflow-, not Snorkel-specific) is saved. You could reload the model outside of Snorkel then. However, you'd need to replace the code that preprocesses the input data.

Overall, if you're training one of the default models in Snorkel, your easiest bet to start is probably just to use Snorkel outside of the Jupyter notebooks, only importing the modules you need for the end modell

Hope this helps! Alex