jaraco / pip-run

pip-run - dynamic dependency loader for Python
MIT License
130 stars 19 forks source link

Improved sophistication of environment strategies #85

Open jaraco opened 9 months ago

jaraco commented 9 months ago

I'd like to consider some possible improvements to PIP_RUN_MODE to make it more configurable and responsive:

consistent voice

Instead of "ephemeral", use "destroy" to match the active voice of "persist". Or maybe prefer "persistent" and "ephemeral" to describe the environment and not the behavior.

Done in #86.

mode parameters

Consider allowing for parameters to the strategy. For example, allow specifying a max age for any enviroment:

mode=persist; max-age=NNNN

(similar to cookie syntax).

new mode - persist resolved dependencies

A new mode "persist resolved" would use PyPI to resolve all dependencies and only then calculate the cache key. This mode could re-use environments when no dependencies change, but be responsive when they do. It too could use parameters to cache the resolution with a TTL.

mode=persist resolved; max-age=NNNN

And maybe then, the default mode should be persist resolved; max-age=86400. This approach would resolve dependencies once a day (for any given spec) and re-use the resolved set when unchanged.

new mode - in-memory install

Imagine - instead of having to assemble expanded installs of packages into some folder on a file system, instead resolving dependencies to wheels and importing modules from those wheels directly from zip files in memory, similar to how a web browser can load javascript dependencies from a URL. This ambitious approach would require developing custom loaders/finders/resource providers and developing facilities for handling non-pure wheels.

Tracked in #89.

bswck commented 6 months ago

I can try to implement the in-memory install mode. Sounds fun 🚀

bswck commented 1 week ago

The demand for new mode - persist resolved dependencies increases with https://github.com/coherent-oss/coherent.test/issues/2. Filed #103 to track.