search

lmstudio-ai / venvstacks

Virtual environment stacks for Python
https://lmstudio-ai.github.io/venvstacks/
MIT License
1 stars 0 forks source link

Run the archive builds in parallel #4

Open ncoghlan opened 4 months ago

ncoghlan commented 4 months ago

With the switch to uv massively speeding up the environment creation processing by running the wheel installs (which are just a zip archive unpacking operation) in parallel, it is notable that the overall bundle build process could be substantially sped up by running the archive builds for each layer in parallel.

Making this switch would require switching to a more sophisticated progress bar implementation that can handle updating multiple bars in parallel (probably tqdm, since pdm already depends on that).

ncoghlan commented 3 months ago

(The precedent in conda-pack suggests that it may even be possible to add multiple files to an archive in parallel, speeding things up even further, but it isn't clear if that's compatible with reproducible archive builds or not)