Open leplatrem opened 7 months ago
I'm a big fan of solution 1, keep things as simple as possible. We only publish one package anyway, right? If anybody wants to make a custom build it should still be just as simple to keep doing that.
Something to consider, with Solution 1
we can't just use pip install kinto && kinto start
to start a quick in-memory storage backend. Because we would require postgresql client libs (ie. compiling psycopg for example). Also in all our repos' CI like kinto-wizard, etc.
Our
requirements.txt
is mainly used as a lock / constraints file. We use it everywhere withpip install -c requirements.txt
(except in the docker file, see PR).pip-compile
is not able to generate a constraints file from all mentioned dependencies inpyproject.toml
(main + extras).pip
does not support extras dependencies in constraints files (that's why we havestrip-extras = true
), and does not support hashes when installing the package locally in editable mode (that's why we havegenerate-hashes = false
). We need editable mode to be able to make changes without having to reinstal the package between two tests runs for example.In #3359, we took this approach:
pyproject.toml
requirements.in
requirements.txt
generated usingpip-compile
.Ideally we would to avoid this duplication, and have one source of truth for all kinds of dependencies. Then have a constraints/lock file for reproducible builds/tests/installations.
Solution #0: status quo
This duplication is fine. The list of package does not change so often.
Solution #1: Get rid of extra dependencies
Simplify everything, and install all dependencies instead of having
monitoring
,postgresql
,memcached
extras.We would still need a distinction for development dependencies, but that's straightforward and doesn't really require pinning.
Solution #2: split into several
*.in
filessetuptools
is able to read*.in
files frompyproject.toml
. That's what we do in kinto-attachmentWe could have:
requirements.in
requirements-postgresql.in
requirements-memcached.in
requirements-monitoring.in
requirements-test.in
requirements-dev.in
The source of truth for the list of dependencies would be these
.in
files.Solution #3: use poetry
poetry
has a simple.Piplock
feature, and has many features around fine-grained installation of extra dependencies and groups.But
poetry
does not support the[project]
section (yet).We would have to move away from setuptools and setuptools-scm, and use a
[tools.poetry]
section.