First-class support for uv.lock?

hynek commented 3 months ago

What's the problem this feature will solve?

Uv added a cross-platform lock format in 0.3.0. I can see this to become a standard for packages to keep CI stable.

Describe the solution you'd like

Given how early it is on uv’s and tox-uv’s life, I could imagine tox-uv to detect the lock file automatically and instead of doing uv pip install it would run uv sync?

It [cs]ould be configurable of course, but this would seem like a great step improvement on general DX.

Alternative Solutions

I mean we can do it by hand or install_commands.

Additional context

https://docs.astral.sh/uv/concepts/projects/

butterlyn commented 3 months ago

I mean we can do it by hand or install_commands.

@hynek As a stopgap measure, could you please share how to do this with install_command? :)

Was unable to figure out a workaround

hynek commented 3 months ago

I cannot. I was/am on my phone and only guessing; sorry. :)

gaborbernat commented 3 months ago

I am not against this. Actually, I would like to see support for this. However, I did not get any chance to look into this release and how it works and what is the feature gap. I might get to it in a month or so, but don't expect anything anytime soon. If someone else has time and can put together a proposal in the meantime, That would be much appreciated.

butterlyn commented 3 months ago

I think it'd be worth waiting for https://github.com/astral-sh/uv/issues/5632 to be actioned, adding uv tools to a project / dependency groups to the lock file. At the moment, the uv lock file doesn't include any developer tool dependencies

Actionb commented 3 months ago

Might have to wait for this https://github.com/astral-sh/uv/issues/5229 as well?

At the moment, uv sync installs into the environment at $PROJECT_ROOT/.venv - not into the currently active environment (i.e. the tox env).

hynek commented 2 months ago

JFTR UV_PROJECT_ENVIRONMENT that allows to point at venvs has shipped

hynek commented 2 months ago

also JFTR, since I looked into it closer: https://github.com/astral-sh/uv/issues/5632 shouldn't be a blocker since it's strictly an add-on.

It's about having dependency groups within --dev, currently there's only one. But you can already have optional dependencies like now and they're locked just fine.

At the moment, the uv lock file doesn't include any developer tool dependencies

That also means that this isn't accurate (anymore?). It locks dev deps just fine, you just can have one group of them.

BarrensZeppelin commented 2 months ago

I use the following configuration to use the uv.lock file:

[testenv]
description = run the tests with pytest
set_env =
    UV_PROJECT_ENVIRONMENT={envdir}
commands_pre =
    uv sync --python {env_python} --locked --inexact
commands = ...

And I use uvx --with tox-uv tox ... to run tox. :slightly_smiling_face:

hynek commented 2 months ago

Cool, something like that I had in mind when I guessed it could be faked for now!

But you probably want --frozen, not --locked because the latter errors out the moment the lock file is out of date which usually is not what you want. The whole point is to have stable builds and handle updates safely.

Does commands_pre run before the package itself is installed? Because then you can save some nanoseconds by passing --no-install-project, too. ;)

hynek commented 2 months ago

Oh and N.B. that uv sync installs all dev deps if you don't pass --no-dev. I'm not sure if you can receive extras in commands_pre? Because getting them installed later using uv pip won't respect the lock file.

BarrensZeppelin commented 2 months ago

But you probably want --frozen, not --locked because the latter errors out the moment the lock file is out of date which usually is not what you want. The whole point is to have stable builds and handle updates safely.

I prefer the error when the lockfile is out of date, but --frozen is also a valid option. :slightly_smiling_face:

Does commands_pre run before the package itself is installed? Because then you can save some nanoseconds by passing --no-install-project, too. ;)

I also use

[testenv]
package = skip

[tool.uv]
package = false

(in pyproject.toml)

So installing the project is of no concern to me.

Oh and N.B. that uv sync installs all dev deps if you don't pass --no-dev. I'm not sure if you can receive extras in commands_pre? Because getting them installed later using uv pip won't respect the lock file.

I do want the uv declared dev dependencies.

uv sync supports the --extra option (which can be supplied multiple times). {extras} in tox is a list, so it won't work out of the box, but maybe it's possible to do some massaging while substituting?

RomainBrault commented 2 months ago

Another solution:

[tox]
min_version = 4.4.0
requires =
    tox-uv>=1.11.3
    uv>=0.4.10

[testenv]
package = skip
commands_pre =
    {work_dir}{/}{provision_tox_env}{/}bin{/}uv export --python={env_python} --locked --quiet --no-emit-project --output-file {env_dir}{/}requirements.txt
    {work_dir}{/}{provision_tox_env}{/}bin{/}uv pip install --python={env_python} --quiet --require-hashes --verify-hashes --strict --requirement {env_dir}{/}requirements.txt
    {work_dir}{/}{provision_tox_env}{/}bin{/}uv pip install --python={env_python} --quiet --no-deps {tox_root}

commands = ...

this use tox auto provisioning to get tox-uv's uv.

Note: I saw that tox auto provisioning use pip and not uv pip. It would be nice to switch to uv pip

gaborbernat commented 2 months ago

Some of my thoughts on the topic. I believe that lock files only makes sense for applications. As such, I am tempted to only support applications. That being said, there can be two types of applications:

installed - the project itself is a library (the business logic is put onto the sys.path via normal or editable install),
inline - the project is not installed, instead setting the PYTHONPATH is how is injected onto the sys.path.

uv today only supports the former variant. In the world of uv everything needs to be installed. uv sync accordingly does two things at once: installs the app and the dependencies. This is causing problems in the world of tox because installing the dependencies and the application are two separate steps within tox. Merging them is not allowed by the existing API. Accordingly, a lock file implementation requires creation of a new environment type, where it is not immediately clear what would:

tox run --pkg-only do 🤔 (perhaps fail with a not support error)?
likely one would have to define a new Python base implementation because packaging is no longer something that can be called separately 🤔 (is an uv implementation detail if the app is packaged as a sdist and installed, as a wheel or some other way). From tox POV, this environment is likely equal to an environment where you have set package=skip. The fact that there is packaging involved at all is really an implementation detail for uv sync.

hynek commented 2 months ago

I believe that lock files only makes sense for applications.

I would softly push back against this. There's plenty of projects that lock their test environments to have stable CIs and having good tooling for that will make that group only grow. For example, I'm close to doing that with attrs since we get constant breakage from Pyright and Mypy updates.

But:

As such, I am tempted to only support applications. That being said, there can be two types of applications:

installed - the project itself is a library (the business logic is put onto the sys.path via normal or editable install),

inline - the project is not installed, instead setting the PYTHONPATH is how is injected onto the sys.path.

It doesn't make a difference. ;) All my apps are packages.

uv today only supports the former variant. In the world of uv everything needs to be installed. uv sync accordingly does two things at once: installs the app and the dependencies.

Unless I misunderstand you, this is not true. To my regret, uv added inline projects in 0.4.0: This release adds first-class support for Python projects that are not designed as Python packages (e.g., web applications, data science projects, etc.).

This is causing problems in the world of tox because installing the dependencies and the application are two separate steps within tox. Merging them is not allowed by the existing API.

This is a solved problem, especially as of uv 0.4.12. You can run uv sync --no-install-project and then just uv sync -- just like in multistage-stage Docker builds: https://hynek.me/articles/docker-uv/

The only problem currently is that uv sync automatically installs dev dependencies but only has 1 type of them. I would argue, that until they grow multiple ones, tox-uv should sync with --no-dev and use extras for test deps as before (e.g., uv sync --no-dev --frozen --extra tests).

gaborbernat commented 2 months ago

I would softly push back against this. There's plenty of projects that lock their test environments to have stable CIs and having good tooling for that will make that group only grow. For example, I'm close to doing that with attrs since we get constant breakage from Pyright and Mypy updates.

My solution is to pin mypy 😊 leave the rest to free flow in my library. But alas, if we support an installable app, we support a library too because from the POV of tox there is no difference really.

Unless I misunderstand you, this is not true. To my regret, uv added inline projects in 0.4.0: This release adds first-class support for Python projects that are not designed as Python packages (e.g., web applications, data science projects, etc.).

Ah, I see, https://docs.astral.sh/uv/concepts/projects/#applications. Alas, if we support the lib variant we also app variant no? As that's the simpler version of it.

gaborbernat commented 2 months ago

Give https://github.com/tox-dev/tox-uv/releases/tag/1.12.0 a go and we can go from there to improve it 👍

gaborbernat commented 2 months ago

https://github.com/tox-dev/tox-uv?tab=readme-ov-file#uvlock-support

jab commented 1 month ago

Followed the new docs and they worked like a charm, thanks @gaborbernat! Things are so much simpler now with a uv lockfile rather than requirements.txt files.

I believe that lock files only makes sense for applications.

Just wanted to push back on this as well. I wrote down why all projects (whether apps or libraries) should use checked-in lockfiles. (Briefly: it's a direct consequence of the "not rocket science" rule, so if we agree that libraries also deserve reliable, deterministic CI, it implies the use of lockfiles.) If you are interested in reading the full rationale, I'd be curious if you find it persuasive.

gaborbernat commented 1 month ago

My counter argument to libraries using lock files is that if you use a lock file and you pin your dependencies, it means that whenever your dependencies break you, you will not know. So yes, your CI will become stable, but it will also mean that you will not know until you refresh your lock file that one of your dependencies broke one of your expectations and now your library is no longer working. I personally prefer my CI to become an early canary, warning me when something breaks for my users.

gaborbernat commented 1 month ago

@jab Note that the latest version of tox now is able to be configured via Toml file. https://tox.wiki/en/4.21.2/config.html#pyproject-toml-native

jab commented 1 month ago

That is a common response, which I addressed in what I wrote: using tooling like dependabot gives you the best of both worlds: you get the early warning when there is a breaking (transitive) dependency update, but without sacrificing properties 1, 2, and 3.

Also, nothing you can do as a library maintainer can prevent applications depending on you from breaking if they aren’t using lockfiles due to some nondeterministic dependency resolution. Ultimately it is the responsibility of every application to lock.

Another way to look at it is, your library’s test suite is itself an application, and therefore needs a lockfile to get the full (and intended) benefits of CI.

gaborbernat commented 1 month ago

The problem with dependabot s that it generates a lot of extra PRs That I need to review and approve ( The amount of time I need to do in my experience is a lot larger than just fixing the CI whenever it breaks every year or so). It also makes the version control history much more convoluted. I do not argue that there's no value in having a lock file. I just think the upsides for me personally are not enough to counteract the downsides and additional work generated. There is no 100% winner, it's just trade-off, ultimately. And in my personal opinion, the trade-off is not worth it, for me personally.

Perhaps when you are maintaining a single library, the amount of extra work is not that significant and you're willing to work with the trade-offs. However, for me, when you maintain 20 projects, you do not want to get 20 PRs every week/month To review and approve when realistically I would need to fix breakages maybe five of them every year. And when I get an email that one of my projects failed on the default branch then it is very clear that it failed. For pull request I have to open up the Web page and check if everything passed or it failed, which is an extra step.

jab commented 1 month ago

I get it. I actively maintain only one library and even for that Dependabot is too noisy. For me the correct tradeoff is to set up a monthly job that does “uv lock --upgrade” (to batch updates together) and send me the PR. This approach allows library maintainers to tune the frequency / noise factor to their own personal circumstances. (I tell you, it’s really nice to be able to go on vacation with my family and not worry I’ll get an email about a nightly CI job failing only because it pulled some new transitive dependency - and I just want the same peace of mind for all library maintainers:)

gaborbernat commented 1 month ago

For me getting that email is just a notification. If I'm not around I'm happy to ack and ignore. When I get back I'll fix it. In the meantime the falling CI is a signal for end users too that something is broken. Also because I keep getting the email daily makes sure I don't miss it once I'm back. Again this is probably different when you maintain 20 vs 1 libraries 😅

jab commented 1 month ago

For me number of libraries maintained does not matter, it’s the developer experience with regard to where the failure appears. With no lock files, users just see the red “main/master is broken” badge on the homepage. It’s indistinguishable from the maintainer leaving main in a broken state due to some change the maintainer actually made. If I’m on vacation for another week, as the maintainer of such a project, this would drive me crazy. On the other hand, with lockfiles, the breakage is contained to the associated “upgrade deps” PR, it’s clearly attributed to that upgrade in the diff, and the build badge for main stays green for users in the meantime.

gaborbernat commented 1 month ago

Hence why I said it's a matter of personal opinion. I prefer not having lock files for libraries. Less work for me, and better dependency failure notification.

tox-dev / tox-uv