Open gerritholl opened 3 years ago
Most users will not notice much, but there will be some changes to the build process.
This would require users to have a modern version of setuptools installed. Also, what specifically would have to change in the build process? Shouldn't the actual commands to do the building be the same?
I've noticed that xarray puts all their stuff in setup.cfg:
https://github.com/pydata/xarray/blob/main/setup.cfg
How can/should this be done in pyproject.toml? Do you have an example?
Actually, setuptools doesn't even support it yet, but build dependencies can be defined in the [build-system]
section of pyproject.toml
, so pip install
will pick those up if needed before doing anything. I don't know how it would work with conda, though. Maybe it's too early, and this issue should be seen as being still blocked.
I was playing around with pyproject.toml
yesterday. With this content for pyproject.toml
:
[project]
name = "trollstorm"
project_name = "trollstorm"
description = "Satellite-based storm research, detection, and nowcasting"
requires-python = ">=3.9"
classifiers = [
"Programming Language :: Python",
"Development Status :: 2 - Pre-Alpha",
"Natural Language :: English",
"Intended Audience :: Science/Research",
"License :: OSI Approved :: GNU Affero General Public License v3 or later (AGPLv3+)",
"Operating System :: OS Independent",
"Topic :: Scientific/Engineering :: Atmospheric Science",
"Topic :: Scientific/Engineering :: Physics",
]
dynamic = ["version"]
readme = "README.rst"
license = "LICENSE"
keywords = ["nowcasting"]
packages = ["trollstorm", "trollstorm.tests"] # replace when setuptools/ppsetuptools supports find
[[project.authors]]
name = "Gerrit Holl"
email = "gerrit.holl@dwd.de"
[project.urls]
repo = "https://github.com/gerritholl/trollstorm"
[project.optional-dependencies]
test = ["pytest", "pytest-cov"]
doc = ["sphinx-pyproject"]
[build-system]
requires = ["ppsetuptools", "setuptools>=42", "toml >= 0.10.1", "setuptools_scm[toml]>=3.4"]
build-backend = "setuptools.build_meta"
[tool.setuptools_scm]
write_to = "trollstorm/version.py"
[tool.pytest.ini_options]
addopts = "-v --cov trollstorm --cov-report term-missing"
testpaths = "tests"
and this content for setup.py
(I don't understand why it's necessary, but it appears to be):
from ppsetuptools import setup
setup()
With this, the package it's installable without pre-installing anything else. Pytest and setuptools_scm already read their configuration from pyproject.toml
, but sphinx and flake8 don't (there exist bridge packages here too, but I found them too obscure). Installing this in a clean conda environment (conda create -n test python=3.9
) with pip install -v
results in pip first picking up the build dependencies and then running setup.py (apparently; I thought it would use the build-backend, but not really).
The verbose install log including stderr is too long to include inline (1844 lines), but includes all the details including the installation of ppsetuptools, which would be needed until setuptools natively supports pyproject.toml, which is probably soon. In either case, the user installing with pip needs to do nothing; I don't know what happens with conda, though. I don't know if I could use the test-dependencies from a GitHub runner directly.
This is really cool! Thanks for the example.
I don't know what happens with conda, though. I don't know if I could use the test-dependencies from a GitHub runner directly.
Do you mean for the conda-forge package? That would still have to be a separate dependency list and that is done manually. The only thing that matters as far as the setuptools/setup.py dependency list is whether or not the conda-forge bot can read it since it makes suggestions based on changes it detects.
For the github runner, are you saying get rid of our conda yaml
that we use in the CI? That would be cool, but since conda packages don't necessarily have to match the PyPI package name I'm not sure we can guarantee this will work forever.
No, I didn't get rid of the conda yaml
file. But I hope that as tools move to supporting pyproject.toml
, we may eventually avoid such duplication of configuration. PEP 621 has only been final for four months, so I suppose projects still need some time to catch up.
That's really nice @gerritholl ! Some time ago I also looked a poetry which looks awesome and supporting pyproject.toml would allow us to use it. And if we do, then there is a tool to convert pyproject.toml to conda format: https://pypi.org/project/poetry2conda/
Feature Request
Is your feature request related to a problem? Please describe.
Satpy package metadata is currently spread and partly duplicated between
pyproject.toml
,setup.cfg
,setup.py
,conf.py
,MANIFEST.IN
, and various tool-specific configuration files.PEP 621 defines how project metadata should be stored in
pyproject.toml
. This PEP was declared final on 2021-03-03. The community is migrating topyproject.toml
, which will deprecatesetup.py
,setup.cfg
, and — eventually — project specific configuration files.Describe the solution you'd like
I would like that Satpy stores as much as possible in
pyproject.toml
. I would like nothing else will be needed anymore.Describe any changes to existing user workflow
Most users will not notice much, but there will be some changes to the build process.
Additional context
Status quo works. Tooling is not entirely ready yet. Setuptools does not yet support pyproject.toml, but ppsetuptools does and probably setuptools will sooner or later. Can be read by pytest already. Hopefully other tools like sphinx and flake8 will follow in migrating to pyproject.toml. Perhaps this issue should wait a bit still.
This issue applies equally to almost all other Pytroll projects.