lukehsiao
commented
6 years ago @dhimmel If you decide to pursue this, you can also refer to Fonduer. We recently made Fonduer pip-installable. Some other things you'll need to watch out for:
- Environment variables, basically these shouldn't need to be there at all, instead, we refactored the code to use proper paths, or have extra parameters that could specify information that was previously done in environment vars
- Making dependencies clear: for example, sqlite, corenlp, postgres, etc. The docs will need to be updated with that information.
- File organization: Right now the tutorials in this repository depend on this repo's organization. You don't want to have tutorials and other unrelated code packaged with snorkel. We recently split out our tutorials into a separate repo in order to be a more accurate representation of how to use Fonduer as a pip package. If you don't want to split them out, you'll just need to make sure that setup.py does the right thing.
I'm sure there's more, but at least those are some things to consider.
dhimmel
stephenbach
It would be make snorkel more portable if it were installable as a package. I'm happy to try to tackle this today and tomorrow (while we're at Stanford Info Lab). The plan would be to create a
setup.pyso snorkel could be installed as a package in a versioned way. This hopefully would reduce the need to set environment variables, which has been causing us headaches.Related content:
treedlibis now pip installable (thanks to @lukehsiao in https://github.com/HazyResearch/treedlib/pull/43)@ajratner and @stephenbach: do you know of any reasons that prevent using snorkel as a package rather than submodule?