search

renovatebot / renovate

Home of the Renovate CLI: Cross-platform Dependency Automation by Mend.io
https://mend.io/renovate
GNU Affero General Public License v3.0
16.48k stars 2.11k forks source link

New python package manager: uv #27841

Open rarkins opened 3 months ago

rarkins commented 3 months ago

New package manager questionnaire

Did you read our documentation on adding a package manager?

Basics

What's the name of the package manager?

uv

What language(s) does this package manager support?

Python

How popular is this package manager?

This is a new manager, but it feels like it will replace pip or poetry

Does this language have other (competing?) package managers?

poetry, pip

What are the big selling points for this package manager?

Detecting package files

What kind of package files, and names, does this package manager use?

Do many users need to extend the fileMatch pattern for custom file names?

Is the fileMatch pattern going to get many "false hits" for files that have nothing to do with package management?

No.

Parsing and Extraction

Can package files have "local" links to each other that need to be resolved?

No

Package file parsing method

The package files should be:

Which format/syntax does the package file use?

How should we parse the package files?

Does the package file have different "types" of dependencies?

Usage standard pep-621 and cli uf pip compile --extra {type}

List all the sources/syntaxes of dependencies that can be extracted

Don't know

Describe which types of dependencies above are supported and which will be implemented in future

Don't know

Versioning

What versioning scheme does the package file(s) use?

classic semantic version - x.y.z

Does this versioning scheme support range constraints, like ^1.0.0 or 1.x?

Lookup

Is a new datasource required?

Will users want (or need to) set a custom host or custom registry for Renovate's lookup?

Where can Renovate find the custom host/registry?

Are there any constraints in the package files that Renovate should use in the lookup procedure?

Don't know

Will users need the ability to configure language or other constraints using Renovate config?

Artifacts

Does the package manager use a lock file or checksum file?

requirements.txt - lock file

Is the locksum or checksum mandatory?

Don't know

If lockfiles or checksums are used: what tool and exact commands should Renovate use to update one (or more) package versions in a dependency file?

Comand not found

Package manager cache

Does the package manager use a cache?

https://github.com/astral-sh/uv?tab=readme-ov-file#dependency-caching

If the package manager uses a cache, how can Renovate control the cache?

uv cache clean - delete .cache directory

Should Renovate keep a cache?

Generating a lockfile from scratch

Renovate can perform "lock file maintenance" by getting the package manager to generate a lockfile from scratch. Can the package manager generate a lockfile from scratch? uv pip compile -o requirements.txt Must be usage override this command, because compile would like execute with parameters --extra tests --extra linters, this group dependencies (dev/prod).

Other

What else should we know about this package manager?

Originally posted by @staners2 in https://github.com/renovatebot/renovate/discussions/27817#discussioncomment-8731017

rarkins commented 3 months ago

This looks like it could be an alternative "processor" in the pep621 manager: https://github.com/renovatebot/renovate/tree/main/lib/modules/manager/pep621/processors

Does uv identify itself either in the pyproject.toml or requirements.txt?

github-actions[bot] commented 3 months ago

Hi there,

Get your discussion fixed faster by creating a minimal reproduction. This means a repository dedicated to reproducing this issue with the minimal dependencies and config possible.

Before we start working on your issue we need to know exactly what's causing the current behavior. A minimal reproduction helps us with this.

To get started, please read our guide on creating a minimal reproduction.

We may close the discussion if you, or someone else, haven't created a minimal reproduction within two weeks. If you need more time, or are stuck, please ask for help or more time in a comment.

Good luck,

The Renovate team

staners2 commented 3 months ago

This looks like it could be an alternative "processor" in the pep621 manager: https://github.com/renovatebot/renovate/tree/main/lib/modules/manager/pep621/processors

Does uv identify itself either in the pyproject.toml or requirements.txt?

Yes. uv usage standard pep621 and identify itself in the pyproject.toml. It creates a lock file (aka requirements.txt, but can be called differently), which I would like to update automatically with changes in pyproject.toml

staners2 commented 3 months ago

This project demonstrate expected behaviour: https://github.com/staners2/uv-renovatebot

rarkins commented 3 months ago

Is there always a 1:1 mapping from pyproject.toml -> requirements.txt? Is the requirements.txt in the same directory?

We need to work out how to locate uv projects, either:

rarkins commented 3 months ago

Reproduction forked to https://github.com/renovate-reproductions/27841

staners2 commented 3 months ago

We create requirements.txt near pyproject.toml. Although somebody may call it differently and somewhere differently store it. Because file requirements.txt described in the format ${dependency_name}=${version} and for download dependencies executed command pip install -r ${file_name} or uv pip sync -n ${file_name}, he may named differently.

I think named file with all dependencies after uv compile should change in renovate.json

We have one requirements.txt for all environment (prod, stage, test), although best practice says split you dependencies for each environment. But standard doesn't allow storing dependencies for each environments in one file requirements.txt

rarkins commented 3 months ago

It seems like if we want to do this properly then we'd fileMatch for the requirements.txt output files and not the pep621 input files, then trace them back to their source.

We'd also want both the pep621 as well as pip_requirements managers to ignore these respective files if the uv manager was processing them.

ods commented 3 months ago

Using pyproject.toml is just one of uv usage scenarios, similar to pip-tools. Actually pyproject.toml doesn't work well with dependencies split into production/development.

rarkins commented 3 months ago

We should maybe then group it as part of pip-tools if it's a drop-in replacement. Then we need to detect when uv was used and not pip-tools, and run it instead

sigma67 commented 1 month ago

It seems the astral.sh folks are focusing on integrating with rye, which now uses uv as its backend for locking and dependency installation

So there's probably significant overlap with #25273 - may be able to avoid two different managers

karfau commented 6 days ago

not sure if this is somehow implicitly covered by the comments above: https://github.com/astral-sh/uv/blob/main/PIP_COMPATIBILITY.md#pip-compile-defaults

By default, uv does not write the compiled requirements to an output file. Instead, uv requires that the user specify an output file explicitly with the -o or --output-file option.

With the options we tend to use for our python projects, renovate ignores files containing -o in the header.

zanieb commented 23 hours ago

👋 just to provide some clarity here, we have a uv pip compile command that should be roughly drop-in for pip-tools's requirements.txt. We also provide additional functionality, like a --universal flag for universal locking in that format.

We are also close to stabilizing a uv lock command which creates a uv.lock file from a pyproject.toml.