airspeed-velocity / asv

Airspeed Velocity: A simple Python benchmarking tool with web-based reporting
https://asv.readthedocs.io/
BSD 3-Clause "New" or "Revised" License
869 stars 180 forks source link

RFC: Resolvers and environment management in ASV #1436

Open HaoZeke opened 2 weeks ago

HaoZeke commented 2 weeks ago

There are now a bunch of front-end (CLI) tools which handle dependency resolution and installation of environments, including rip, rye (both already dropped out[^2]), and uv.

A key issue is that most of these tools (including mamba) don't actually expose the functionality which asv really needs[^1], that is the dependency resolution / a Python API (e.g. see https://github.com/astral-sh/uv/issues/7345) itself. This is accomplished via pubgrub-rs in uv, resolvo in pixi / rip and the venerable libsolv in mamba.

[^1]: e.g. the trouble with the libmambapy dependency which needed a new release since part of the API was removed. [^2]: e.g. this comment

Somewhat related are these issues around the ecosystem:

Basically, moving forward, it seems like the best approach is to generate bindings to uv internals like the dependency resolution while extracting the environment management out further. Or by using py-rattler (here). Although the CycloneDX SBOM might also be of interest as an intermediate file format ^3.

Another option is to use subprocess to drive calls to uv or something similar, but in the long run this is more problematic, since parsing CLI is never a great idea, and also leaves the project vulnerable to changes in the in internal lock file representation.

HaoZeke commented 2 weeks ago

For some more context, this matters to asv because when run with existing environments, we still need to know the changes to versions relative to the last known configuration.