search

juftin / hatch-pip-compile

hatch plugin to use pip-compile (or uv) to manage project dependencies and lockfiles
http://juftin.com/hatch-pip-compile/
MIT License
74 stars 3 forks source link

✨ make pip-tools a dependency #42

Closed oprypin closed 9 months ago

oprypin commented 9 months ago

Rather than installing pip-tools into each destination environment, make it a dependency and call it programmatically

Note: I'm not sure if this doesn't cause some complications/regressions. I am confident that it should be a dependency. I'd prefer still calling it as a CLI but in the original environment somehow. But actually I didn't find how to refer to it in the original environment, so instead I did it by using it through an import.

juftin commented 9 months ago

Unfortunately pip-compile must be called from the virtual environment it is creating a lockfile for otherwise you could end up with with lockfile issues.

Take this commit as an example: https://github.com/juftin/hatch-pip-compile/commit/c54d5dc1095af096baae2df4f8790d989a617385. I add a pandas>=2.14 dependency to the matrix environment and regenerate the 3.8 matrix lockfile. Even though the lockfile looks like it was generated with Python 3.8, it was actually generated by 3.11. I know this because Pandas >=2.14 requires Python >= 3.9 and pip-compile can't handle that requirement in a 3.8 environment. 

rm requirements/requirements-matrix.py3.8.txt
hatch run +py=3.8 matrix:cov

So the lockfile we're left with is incompatible with 3.8 even though it was created for that environment. On the installation step it raises the following error:

──────────────────────────────────────────────────────────────────────────────────────────────────── matrix.py3.8 ────────────────────────────────────────────────────────────────────────────────────────────────────
ERROR: Ignored the following versions that require a different python version: 1.25.0 Requires-Python >=3.9; 1.25.0rc1 Requires-Python >=3.9; 1.25.1 Requires-Python >=3.9; 1.25.2 Requires-Python >=3.9; 1.26.0 Requires-Python <3.13,>=3.9; 1.26.0b1 Requires-Python <3.13,>=3.9; 1.26.0rc1 Requires-Python <3.13,>=3.9; 1.26.1 Requires-Python <3.13,>=3.9; 1.26.2 Requires-Python >=3.9
ERROR: Could not find a version that satisfies the requirement numpy==1.26.2 (from versions: 1.3.0, 1.4.1, 1.5.0, 1.5.1, 1.6.0, 1.6.1, 1.6.2, 1.7.0, 1.7.1, 1.7.2, 1.8.0, 1.8.1, 1.8.2, 1.9.0, 1.9.1, 1.9.2, 1.9.3, 1.10.0.post2, 1.10.1, 1.10.2, 1.10.4, 1.11.0, 1.11.1, 1.11.2, 1.11.3, 1.12.0, 1.12.1, 1.13.0, 1.13.1, 1.13.3, 1.14.0, 1.14.1, 1.14.2, 1.14.3, 1.14.4, 1.14.5, 1.14.6, 1.15.0, 1.15.1, 1.15.2, 1.15.3, 1.15.4, 1.16.0, 1.16.1, 1.16.2, 1.16.3, 1.16.4, 1.16.5, 1.16.6, 1.17.0, 1.17.1, 1.17.2, 1.17.3, 1.17.4, 1.17.5, 1.18.0, 1.18.1, 1.18.2, 1.18.3, 1.18.4, 1.18.5, 1.19.0, 1.19.1, 1.19.2, 1.19.3, 1.19.4, 1.19.5, 1.20.0, 1.20.1, 1.20.2, 1.20.3, 1.21.0, 1.21.1, 1.21.2, 1.21.3, 1.21.4, 1.21.5, 1.21.6, 1.22.0, 1.22.1, 1.22.2, 1.22.3, 1.22.4, 1.23.0rc1, 1.23.0rc2, 1.23.0rc3, 1.23.0, 1.23.1, 1.23.2, 1.23.3, 1.23.4, 1.23.5, 1.24.0rc1, 1.24.0rc2, 1.24.0, 1.24.1, 1.24.2, 1.24.3, 1.24.4)
ERROR: No matching distribution found for numpy==1.26.2

pip-sync on the other hand has a --python-executable argument that does make it compatible with use between different Python interpreters: https://pip-tools.readthedocs.io/en/latest/cli/pip-sync/

oprypin commented 9 months ago

Ah yes of course, thanks for the explanation :+1: