Release flowdec on pypi

[russellb]

Instead of having install instructions that require cloning the source using git and then installing via pip, flowdec could be released to pypi so the install instructions become just "pip install flowdec".

Roughly, the steps would just be tagging a release in git, then uploading a package of that release to pypi. There are some instructions here: https://packaging.python.org/tutorials/packaging-projects/

[hammer]

pip is cool but hasn't there been a lot of change in the Python packaging ecosystem recently? PEP 517, PEP 518, flit, poetry, pipenv, etc. And of course, conda! What's the current best practice?

[russellb]

I don't know how cool pip is, really. :-)

AFAIK though, pip (the tool) and pypi (the package archive) are still the base, de-facto way to publish Python packages. It seems like the first place you'd publish a Python library. My personal pypi experience has mostly been in two categories: OpenStack (many, many Python components, lots of automation involved in building and publishing packages, not an example to look at) and the Python lib for OVS, where we use a pretty vanilla Python setup.py / pypi publishing setup. I just run something like "python setup.py sdist upload" to push updates.

Afterwards, there may be one or more other places it makes sense to distribute the code. For example, the various linux distributions re-package Python components in their own native packaging mechanisms. Some of that seems to be losing momentum to just building container images using the native language tooling.

I wasn't familiar with Anaconda (or conda) before seeing it mentioned in the cytokit repo somewhere, and then your mention here. I understand the value of a curated set of packages tested together. It seems to accomplish that in part with a new packaging and distribution approach, which I'm not sure the world needed. I shouldn't judge it on such a quick look.

Even with conda, it seems those packages need to be sourced from somewhere, and the first example I saw was how to create a conda package from a package on pypi, so maybe it still makes sense to publish there first?

Maybe generalized, the way I'd think about this:

Goal: make the software easy to acquire by the people who want it.

To achieve that, I'd be looking at a couple of places:

1) The native tooling for the language ecosystem in question. For Python, someone will expect to be able to "pip install" something.

2) Considering the domain, where else would someone want to get the software from? Traditional Linux distributions commonly service this purpose, but more specialized software distributions (like Anaconda, it seems) also fit this category for me.

... I just thought it might be nice to change "git clone .../flowdec && cd flowdec && pip install ." with "pip install flowdec" in a few documents. :-)

[hammer]

Ha top story on HN today: Python's New Package Landscape (from May). There's...a lot going on.

[russellb]

Nice, thanks for sharing that write up!

On Thu, Oct 18, 2018 at 10:23 AM Jeff Hammerbacher notifications@github.com wrote:

Ha top story on HN today: Python's New Package Landscape http://andrewsforge.com/article/python-new-package-landscape/ (from May). There's...a lot going on.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/hammerlab/flowdec/issues/8#issuecomment-431028611, or mute the thread https://github.com/notifications/unsubscribe-auth/AAS4Cm_QpY3fJLAOguGZ9zn1ALVrd69Mks5umI7hgaJpZM4XdYVG .

-- Russell Bryant

[eric-czech]

To get rolling on this then, I uploaded an initial version at https://pypi.org/project/flowdec/0.0.1/ and updated the install docs.

@russellb do you have any recommendations on how to work those releases into the CI process? I just did that upload manually (i.e. python setup.py sdist upload) but I'm guessing there's a better way to do it in conjunction with travis and tox. Let me know if you have any thoughts there?

[armish]

@eric-czech: you just need to configure the deploy section of your .travis.yml and save your (or another dummy account's) password as a secret on Travis. AFAIK, the process was pretty straightforward: https://docs.travis-ci.com/user/deployment/pypi

The only thing that matters most is to decide when to deploy: each push to master, each push to a determined branch, or whenever you push a new tag... My favorite is deploying new tags: https://docs.travis-ci.com/user/deployment/pypi/#deploying-tags

Here is an example configuration for continuous deployment to PyPI: https://github.com/openvax/vaxrank/blob/master/.travis.yml#L89-L97

[eric-czech]

Thanks @armish !

I set that up (.travis.yml) and will use the tagged commits to trigger the uploads from now on. Also, I added a shared CHS-specific pypi user for the lab and added you to it. The credentials are in 1Password and I believe you should have gotten a verification email.