Packaging and publishing

[amotl]

Introduction

@gutzbenj and I recently talked about introducing appropriate packaging for this fine project and finally pushing the result to PyPI.

Naming things

I found python_dwd a bit quirky, so I proposed a different, more fluid-sounding name for it: "pydwd". This follows the naming scheme of many popular packages on PyPI where the py prefix is used to associate the language thing (Python) with a binding to another thing (here: DWD). As the name is not yet taken on PyPI, we should go for it.

Versioning

When there are no objections, we might want to bump down to 0.1.0 and then progressively increase the version number while we go. Also, properly maintaining the changelog will do no harm.

Release pipeline / Continuous delivery

While I used a combination of Makefile targets together with bumpversion to cut new releases within many other projects from my pen, @gutzbenj aims for integrating the whole thing through GitHub CI / GitHub Actions. While I am not fluid with these things yet, I totally second that approach. Kudos to @meteoDaniel for already starting this matter.

Using modern standards for describing project metadata

Associating metadata with Python projects has a long history in form of a more or less bumpy road, it looks like the most modern thing is deprecating setup.py completely and - after setup.cfg was a thing for some time, now using a pyproject.toml file seems to be en vogue. Just using requirements.txt is way not enough as this only outlines the dependencies of the project without taking care of the other metadata required to describe the project thoroughly.

In order to learn more about that, reading PEP-517, PEP-518 and PEP-621 makes sense. While this is also new to me, we should definitively go for it.

Using modern tools for project management

Notwithstanding the above, I want to note that Poetry has gained a huge momentum these days for managing Python projects and is compliant with PEP-517, see https://python-poetry.org/docs/pyproject/.

Test automation

While I also mentioned Tox for automating testing while talking to Benjamin recently, it looks like Nox is the new kid on the block in this field. We should use it.

The plan

The fine article Hypermodern Python Chapter 6: CI/CD by Claudio Jolowicz thankfully outlines a blueprint how to combine GitHub Actions with Poetry and Nox and illuminates many aspects around that workflow which leaves nothing to wish for.

[amotl]

Outcome so far

I've made progress into the pyproject.toml / Poetry thing through #86 and it works very well for me. Please note that I haven't committed the poetry.lock file yet but we should definitively add it to the repository. More about the background of this is outlined within [1].

[1] https://python-poetry.org/docs/basic-usage/#installing-dependencies

Walkthrough

These steps will get you started.

# Acquire sources
git clone https://github.com/earthobservations/python_dwd
cd python_dwd

# Install dependencies
poetry install

# Run tests
poetry run pytest

# Invoke comand line tool
poetry shell
dwd --help

Please note that installing dependency packages will not poison your system Python because Poetry will transparently create and use a virtualenv when invoking poetry install, like

$ poetry env list --full-path
/Users/amo/Library/Caches/pypoetry/virtualenvs/python-dwd-vMipSWwx-py3.8 (Activated)

[meteoDaniel]

@amotl could you summarize the benefits of poetry and nox in a few small key points? Beeing the new kid in the block would not mean that I dirrectly buy stuff from him or her. :stuck_out_tongue_winking_eye:

[gutzbenj]

Please note that installing dependency packages will not poison your system Python because Poetry will transparently create and use a virtualenv when invoking poetry install, like

$ poetry env list --full-path
/Users/amo/Library/Caches/pypoetry/virtualenvs/python-dwd-vMipSWwx-py3.8 (Activated)

Am I getting you right, that even when I install the library in an existing environment, it would still create its own environment in the environment and source its dependencies from there?

And would that work in the same way when invoking pip install python_dwd from PyPi later on?

[amotl]

Hi again,

@meteoDaniel: The benefit of Poetry is that it covers the whole lifecycle of working with a Python package in development mode, i.e. installing dependencies into a dedicated virtualenv without further ado as well as building sdist/wheel packages, version bumping and uploading them to PyPI. Otherwise, we would have to assemble respective things through other tools as I did before with other projects from my pen.

Nox is a command-line tool that automates testing in multiple Python environments, similar to tox. Unlike tox, Nox uses a standard Python file for configuration. tox is the de-facto standard for managing multiple Python test environments, and is the direct spiritual ancestor to Nox.

@gutzbenj: Installing Python packages through pip will work the same way as before. Building an sdist/wheel package locally and installing it into a manually created virtualenv outlines this procedure you are welcome to reproduce:

Build sdist and wheel packages

poetry build

Install into virtualenv using pip

source .venv3/bin/activate
pip install dist/python_dwd-0.1.0-py3-none-any.whl
dwd --help

With kind regards, Andreas.

[amotl]

Of course, sharing this in this context is also a must. ;]

-- https://xkcd.com/1987/

[gutzbenj]

I checked again for the new name pydwd and unfortunately found a Github repo with that name. Thus I suggest to rename the library to wetterdienst, which I found to be very clean and more appropriate for future plans to integrate more weather services within this library.

[amotl]

I like the name Wetterdienst and look forward to bring Autobahn into the mix some day.

[amotl]

Wetterdienst 0.1.0 has just been released on PyPI, see [1]. Thanks a bunch, Benjamin!

[1] https://pypi.org/project/wetterdienst/