Adjust semantics of `requirements.txt`

[amotl]

Report

Currently, the export to requirements.txt includes development dependencies. Is it intended like this?

Thoughts

If the Anaconda packages will derive its runtime dependencies from this list, maybe we should change it? See also https://github.com/conda-forge/wetterdienst-feedstock/issues/57.

Proposal

In order to provide a corresponding file including the development dependencies, we can use a different file name, like requirements-sandbox.txt. In this way, requirements.txt would be more lean by exclusively listing runtime requirements.

export_requirements_runtime = "poetry export --without-hashes --output requirements.txt"
export_requirements_sandbox = "poetry export --without-hashes --with=dev,test --output requirements-sandbox.txt"

_Originally posted by @amotl in https://github.com/earthobservations/wetterdienst/pull/788#discussion_r1027291326_

[gutzbenj]

My idea was to export to requirements.txt to get the dependency graph of Github because it currently only supports this format. That's why also dev libs should be included. However also currently we are reusing the requirements for building docs environment on RTD because it is the only way to get a full installation for our docs.

[amotl]

Thanks for bringing me up to speed on this topic.

Thoughts

a) To get the dependency graph of Github because it currently only supports this format. That's why also dev libs should be included.

I agree. The sweet thing is, GitHub's Dependency graph can pick this up from all of setup.py, and requirements{-*}.txt files. See, for example, ^1.

b) We are also using the requirements.txt for building docs environment on RTD because it is the only way to get a full installation for our docs.

I expected that. Would it be possible to use a dedicated requirements-docs.txt for that purpose?

New proposal

export_requirements_dev = "poetry export --without-hashes --with=dev,test --output requirements-dev.txt"
export_requirements_docs = "poetry export --without-hashes --with=docs --output requirements-docs.txt"
export_requirements_runtime = "poetry export --without-hashes --output requirements.txt"

[gutzbenj]

I just checked the details again and Github seems to support pyproject.toml now! We can drop requirements.txt again but we'll have to keep docs/requirements.txt to configure the sphinx final setup (only basic dependencies, sphinx version and setuptools version).

[amotl]

I just checked the details again and Github seems to support pyproject.toml now.

Sweet.

We can drop requirements.txt again but we'll have to keep docs/requirements.txt.

It looks like we don't have this file in the repository. Am I missing something?

[gutzbenj]

We dont have it but we'll have to create it. When running the poetry install stuff for RTD somehow afterwards RTD itself installs its own sphinx version (although we had already installed it with poetry). However if we create the docs/requirements.txt there we can just define which sphinx version will be installed after the RTD sphinx installation.

[amotl]

When running the poetry install stuff for RTD somehow afterwards RTD itself installs its own sphinx version.

I've just swapped order and used the post_install job, instead of post_create_environment, as advertised.

-- https://docs.readthedocs.io/en/stable/build-customization.html#extend-the-build-process

[amotl]

With #794, we decided to stop exporting any requirements{-*}.txt files, because a) RTD supports Poetry installations well, and b) GitHub's Dependency graph can pick packages from pyproject.toml now.