search

jaraco / pip-run

pip-run - dynamic dependency loader for Python
MIT License
135 stars 20 forks source link

New mode - persist resolved dependencies #103

Open bswck opened 3 months ago

bswck commented 3 months ago

As described in #85.

bswck commented 3 months ago

The new mode should hash resolved dependency trees.

My first idea was to hash the output of pip install [deps...] --silent --report - --dry-run --ignore-installed. However, this isn't the best approach in the long run—https://github.com/jaraco/pip-run/issues/100 suggests the option of relying on uv for installing packages. To be able to use different tools for sole package resolution in the future, it would be ideal to simply use lockfile content hashes as keys.

bswck commented 3 months ago

Lockfiles have various formats. In https://github.com/bswck/lockfiles-overview, I'll try to summarize differences between lockfiles produced by top tools and try to find their "GCD" to describe a minimal hashable dependency tree.

As of now, I'm thinking that packages with versions and hashes should do the thing.

jaraco commented 3 months ago

Other things to consider - I'm not sure how lockfiles work with local disk or URL resources. I've been impressed with what tox has done lately to detect when dependencies have changed (and then make minimal adjustments). For example, when a dependency is removed, it will re-build the environment (invalidate the cache), but if a dependency is bumped or downgraded, it only re-installs that one dependency over the existing environment. It might be useful to examine how they do that.

jaraco commented 2 months ago

100 suggests the option of relying on uv for installing packages

Don't let this be a hard blocker. If there's an elegant solution using just pip, it's okay to implement that and only have the feature available on pip until uv can provide a comparable interface.