search

astral-sh / uv

An extremely fast Python package and project manager, written in Rust.
https://docs.astral.sh/uv
Apache License 2.0
25.72k stars 750 forks source link

RFC: Tool management #3560

Open zanieb opened 6 months ago

zanieb commented 6 months ago

This is a tracking issue and design for management of "tools" in uv, similar to pipx (see https://github.com/astral-sh/uv/issues/1173).

If this design does not fulfill your use-case, let us know in a comment describing what's missing. Please don't comment asking for status updates or a timeline — we'll be working on this actively and everything will be linked to this issue.

👋 Thanks for your interest in this design! I'll be editing this post to reflect the state of discussion, it will not be a static document

Roadmap

Overview

Tools are packages that provide executable entry points.

Tools may be installed, allowing their executable to be used without activation of an environment, e.g. uv tool install ruff then ruff is available. Installation of a tool allows a developer to use the tool across all of their projects. Each tool is installed into an isolated Python environment that is abstracted from the user.

Tools may be added to a project, allowing their executable to be used via uv run, uv tool run, and detected by other tools such as IDEs. Adding a tool to a project allows developers to use a shared environment while working on a specific project. Adding a tool to a project differs from adding a development dependency in that the tool environment is isolated from the project’s environment by default. Tools in a project cannot depend on each other.

The environment used to provide the tool is intended to be abstracted from the user. However, in some cases the user may need to install multiple packages to provide the tool they need e.g. a main package and a plugin. This must be done at declaration or install time, editing tool environments is not supported.

The following commands would be provided

Running tools

Tools can be run with uv tool run. This command will choose the appropriate version of the tool to run based on the working directory. For example, if in a project that contains configuration for the tool, the project's version of the tool will be used. Otherwise, the tool installed at the user level will be used. If the tool is not in the project or installed, the latest cached version of the tool will be used. If the tool is not cached, the latest version will be added to the cache. The --isolated flag can be used to opt-in to using the latest cached version instead of project or installed versions. The --latest flag can be used to force a check for a newer version (this differs from --no-cache in that the cache can still be used for the tool if it is the latest version and for the tool's dependencies). The user can opt-in to a specific version of the tool with <name>@<version> syntax, this will bypass different project or installed versions.

A uvx alias will be provided to run tools. Notably, this would be an alias for uv tool run not uv tool — this could be a point of confusion for users transition from pipx but avoids the need to type uvx run. If we're providing an alias, it should be for the most-used command not for the broader uv tool interface. This matches tools in other ecosystems, like npx.

Additional dependencies may be requested when running a tool, e.g. with a --with <spec> option. These will be installed in the tool's environment. If an installed tool's environment does not contain the request, a new, ephemeral environment will be created to fulfill the invocation — the installed tool's environment will not be modified. By default, the entry points provided by additional dependencies are not available. A flag may be provided to opt-in to this, e.g. pipx provides a --include-deps option.

Unlike uv run, the command name and requirement name are assumed to match e.g. uvx black will run black with an implicit requirement on the package black. Users may opt-out of this implicit requirement by providing a different package name e.g. with uvx --package foo -- bar.

Examples:

# Run a tool command in an isolated environment
uvx ruff check
uv tool run ruff check

# Avoid ambiguity with flags
uv tool run -- ruff --help

# Request a specific version
uvx -v 0.2.0 ruff check
uvx ruff@0.2.0 check

# Request an additional dependency
uvx --with ruff-plugin -- ruff check

# Run a command provided by a package with a different name
uvx --package foo -- bar

# Force the latest version to be used instead of an older installed or cached version
uvx --latest ruff

# Ignore project and installed tools
uvx --isolated ruff

Installing tools

Tools can be installed with uv tool install. Tools can be installed into a user or system namespace, though the system namespace is more difficult to manage and should be considered a secondary objective during implementation. An installed tool is available on the PATH and usable from a terminal without activation of an environment. All entry points provided by the tool package are included by default. The user may opt to include a subset of entry points. If the entry points conflict with other managed tools at the same scope, installation should fail. If the entry points conflict with other executable names, a warning should be displayed during installation.

Similar to uv tool run, additional dependencies may be requested and the package may be manually specified.

The requirement name is used as a "key" for identifying the tool in subsequent interactions e.g. uninstall and upgrade. This behavior may be confusing, if a user requests an operation on a command name that belongs to a requirement. In this case, a hint should be provided explaining the relationship.

When installing a tool, uv will create an entry in its configuration directory to track tool state. This entry is considered the source of truth for the tool's environment.

When installing a tool, a Python environment must be created for its package and dependencies. This environment should be entirely abstracted from the user. The environment should be created in a uv application data directory (platform dependent path).

When installing a tool, its entry points must be linked to executables in a directory on the user's PATH. Multiple candidate directories may be considered with a preference for a directory present in the PATH. If none of the candidates are present in the PATH, the XDG standard directory should be used and a warning should be provided to the user prompting them to add it to the PATH. The user may provide an alternative installation path e.g. --root <path> (as in cargo). This may be preferable to a --system flag.

As described in uv tool run, by default, the entry points provided by additional dependencies are not installed. A flag may be provided to opt-in to installation of all entry points in the dependency tree, e.g. pipx provides a --include-deps option.

Each entry point is added to the directory with its normal name, and name@<version>. This suffix allows tools with conflicting versions to be used across different scopes. The user may customize this suffix with a --suffix flag. If a custom suffix is provided, we will not include the plain name executable during installation.

When specifying suffixes, the @ is implied and will be included if the suffix begins with an alphanumeric character — if included by the user it will be trimmed for compatibility with pipx (which requires it to be included explicitly by the user). Leading characters such as - or _ will drop the implied @ allowing customization of suffixes. This behavior is not considered necessary for the initial implementation of this feature.

uv does not allow installation of multiple tools in the same scope with the same key. If installation of multiple versions of a tool is desired, a suffix must be used. If the user attempts to install another version of an already-installed tool, the existing tool will be replaced. If the existing tool installation contains dependency requests that are not requested in the invocation that would replace it, an error should be raised instead.

Examples:

# Install a tool into the user bin
uv tool install ruff

# Run an installed tool
ruff check

# Install a tool at a specific verison
uv tool install ruff==0.2.0
# Maybe: Allow pins with the same syntax as you would run with
uv tool install ruff@0.2.0

# Install a tool into the system bin
uv tool install --system ruff

# By default, we use the latest installed and compatible Python version for the tool.
# Install a tool with a custom Python version
uv tool install --python 3.8 ruff

# Install a tool with a subset of binaries
uv tool install --system foobar --bin foo

Tools in a project

Tools can be added to a project with uv tool add. This is loosely an alternative to dependency groups — many uses of development dependency groups are for managing project-level tools. We may need to adjust this design depending on the status of PEP 735. This is likely the last part of the design we would implement, as it is the highest risk.

This operation requires a pyproject.toml to be available.

A user may opt-in to having their project’s dependencies available in the tool environment. However, tools may never share environments with each other.

Adding a tool to a project should:

Running uv sync should ensure that all project-level tools are available.

Project-level tools are not added to the PATH automatically. Tool executables must be invoked with uv tool run, uv run, or in an activated environment. uv run should detect and respect tool executables in the project. However, entry points provided by packages in uv run should take precedence over tools. Placing tools into the virtual environment's bin should allow other tools like IDEs to discover the entry points.

As described in uv tool install, suffixes must be provided for tools if multiple versions are needed in the project.

Examples:

# Add a tool to the project
uv tool add ruff

# Add a tool with a specific version to the project
uv tool add ruff==0.3.0

# Add a tool with an extra dependency
uv tool add --with ruff-plugin ruff==0.3.0

# Add a tool to the project that requires the project dependencies at runtime
uv tool add pytest --with-project

Upgrading tools

Tools can be upgraded to newer version with uv tool upgrade.

The scope will be determined by the working directory. If in a project, this will upgrade the tools in the project. Otherwise, this will upgrade installed tools. The user may explicitly select a scope with a --project flag (which will fail if not in a project) or a --user flag. The --system flag is required to upgrade system tools.

If no tool name is provided, the user will be prompted to confirm an upgrade of all tools. If one or more tool names are provided, we will upgrade the given subset. If any of the given names are not found, we should throw an error.

The uv tool entry will be consulted to determine the requirement specification originally used for a tool. By default, will only be upgraded within the specification e.g. if the tool is pinned to a version, it will not be upgraded or if the tool has an upper bound it will not exceed it during upgrades. A flag may be provided to bump pinned packages, but the design of this behavior and its interaction with package specifications with upper bounds is outside the scope of this design and consequently deferred. In the interim, it may be useful to provide a command to view outdated tools or tools that would be upgraded if the restrictions were loosened.

Examples:

# Upgrade a tool
uv tool upgrade black

# Upgrade a tool in the user scope (while in a project)
uv tool upgrade black --user

# Upgrade all tools in the project or user scope
uv tool upgrade

# Upgrade all tools in the system scope
uv tool upgrade --system

Open questions

  1. Should we infer the package name from the command name or vice-versa?
  2. How should we alleviate confusion around the alias, especially w.r.t. mistakes like uvx install?
  3. How do users request their project be included with uv tool add?
  4. How does upgrading work for constrained packages?
  5. How should we provide re-installation of tools? Does it deserve a dedicated command?
  6. Should we allow alternative suffixes to @?
  7. What's the recommended worfklow for adding dependencies to an already-added or installed tool?
zanieb commented 6 months ago

As an aside, I'm a little uncertain about providing two interfaces i.e. uv run and uv tool run/uvx. Unfortunately it feels necessary to have two different interfaces to provide reasonable default semantics for these separate use-cases.

JelleZijlstra commented 6 months ago

A couple of things I noticed on reading:

cjolowicz commented 5 months ago
wlritchi commented 5 months ago

Looks great! Two questions:

  1. As a user, why might I prefer to use uv tool add rather than adding the tool to the project's dev dependencies?

  2. My workflow combines pipx with pyenv for managing available Python versions. If I'm installing a tool that needs a particular Python version, so far I've been doing something like this:

    pyenv install 3.11
pyenv global 3.11
pipx install my-tool
pyenv global 3.12  # reset global version

    Notably, this means that running python3.11 now emits an error:

    pyenv: python3.11: command not found

The `python3.11' command exists in these Python versions:
  3.11.6
  3.11.7
  3.11.7/envs/some-venv

Note: See 'pyenv help global' for tips on allowing both
      python2 and python3 to be found.

    Yet the shim installed for my-tool continues to work since, in the tool's venv, python is symlinked to /home/wlritchi/.pyenv/versions/3.11.7/bin/python3.11. Will uv tool install --python 3.11 my-tool also pin the Python installation allowing a similar use case, or otherwise provide a way to use Python versions that aren't on the PATH?

T-256 commented 5 months ago

The following commands would be provided

  • uv tool run: Run a tool without installation
  • uv tool install: Install a tool, allowing usage in a terminal
  • uv tool uninstall: Remove an installed tool
  • uv tool add: Add a tool to the current project
  • uv tool remove: Remove a tool from the current project
  • uv tool upgrade: Upgrade tools (either all or the specified tool)
  • uv tool list: List available tools
  • uv tool show: Show details about a tool

For consistency, I suggest using uv pip interface which doesn't have add, remove and upgrade. we could have a flag for them in install/uninstall instead of having them as separated subcommands. (Or do you have plan to update uv pip interface to include add, remove and upgrade?)

zanieb commented 5 months ago

Thanks for the comments everyone, they're greatly appreciated.

I'm not too clear on the distinction between the add/remove and install/uninstall commands.

The idea is that uv tool add adds a tool to your project in the same way that uv add will add a dependency to your project. I think once we introduce uv add this will be a bit more intuitive. However, I agree that it's still not quite clear what is a "global" operation and what is a "project" operation. The design suggests --project, --user, and --system flags to adjust the scope of operations. It's possible that separate top-level commands would make sense, but I think I prefer uv tool add and uv add to e.g. uv project add-tool and uv project add-dep or uv project add --tool ... and uv project add --dep .... I think we're already making some progress by omitting a top-level uv install command, which is quite ambiguous but exists in other tools, e.g., cargo and poetry.

It would be good to specify those a bit more. Like how do we determine the system bin across platforms?

I agree that system level installations would need to be specified more, basically figuring out where to put things. I don't think this is particularly clear yet. On each platform, we need a list of candidate directories, filtered by locations it would not be surprising if we wrote to, that we have write access to, and are on the current PATH. This is work that I'm attempting to push into the future. It's not quite critical and I'd rather focus on getting the rest of the implementation out there. Hopefully providing an escape hatch like --root or --install-directory allowing installation to arbitrary locations would buy us time here.

How does a project-level install with project dependencies differ from a development dependency?

This is a great question and highlights how intertwined tool management and development dependencies are. The --with-project flag is intended to allow you to use tools that require your project's dependency tree in an isolated environment. In contrast, development dependencies are synced to the default project environment when you are working on the project. Development dependencies are available as importable libraries in your code, whereas tools are intended to operate on your code via an executable entry point. For example, mypy would be a tool you'd install with --with-project but a library you only import in tests would be a development dependency. A package like pytest lives in a bit of a gray area, but would probably need to be a development dependency, e.g. so you get proper type hints in your IDE.

Adding plugins to an existing tool over time is an important use case.

This will definitely be supported by extending the entry for the tool in a pyproject.toml. For tools outside of projects, I'm hesitant. I do see it being a pain to re-specify all of the requirements and perform a new installation to add a single dependency. You could edit the ~/.uv/tools entry but maybe that's not friendly enough. Perhaps not having an inject command would encourage declarative definitions of tool environments (which is generally a goal of environment management in uv). My qualm with inject might be rooted in its exposure of the virtual environment to the user. I think I'll need to consider this a bit further. If we add a command for this, it should work for both project and global tools.

Why doesn't the usual entry-point mechanism (e.g. shebangs on Unix) work here? How is this different from user-level installs?

Just to expand a bit on the difference here, we do not want to change the PATH based on the current working directory and we don't want to place shims on the PATH for project-level installations. This makes project-level tools different from user-level installs — they are not on the PATH by default. In order to run a project-level tool, we suggest using uvx or uv run. However, this means that project-level tools are not discoverable by things like IDEs. In order to make them discoverable, we'll insert them into the virtual environment's bin directory.

Presumably, as you mention, a shebang can point to the proper tool environment. I guess that's how globally installed executables will be implemented as well :) I hadn't thought much about how we want entry points to work yet, but yeah of course it's fine running an executable that needs a virtual environment that differs from the current one. I'll update the document accordingly. Thanks!

Why might I prefer to use uv tool add rather than adding the tool to the project's dev dependencies?

A very simple example is that the tool's dependencies conflicts with your project's dependencies. We might also not support multiple development dependency groups as in, e.g., cargo vs poetry. If so, then this would be a way to define a separate environment for a "development dependency".

If I'm installing a tool that needs a particular Python version...

There are a couple parts to this, but the short answer is: yes, we hope to solve this. I think there's a bit of lurking complexity in when symbolic links are resolved and pyenv shims — I'm not sure I can guarantee that we will resolve that specific issue, there are similar issues already, i.e., #1795. However, we do intend to perform toolchain management (i.e. install and manage Python for you) and we've invested a lot of time into proper discovery of existing Python interpreters. For both of those cases, we'll need to handle upgrades and switched Python versions. Ideally you'll be able to request a Python version for you tool and it'll just work.

I appreciate that you highlighted this use-case though, we'll definitely need to be careful about interpreter resolution for tool environments.

I suggest using uv pip interface...

We won't be using uv pip for new workflows, it is intended as a plumbing and compatibility layer. All of our new commands will be outside of the uv pip namespace.

CharString commented 5 months ago
  1. For me, the biggest time saver is pipx reinstall it allows me to quickly reinstall after a python interpreter update. But maybe the sync command can do that, or upgrade.
zanieb commented 5 months ago

The biggest time saver is pipx reinstall it allows me to quickly reinstall after a python interpreter update

Yeah I don't specify this command because I'm unsure if it's going to be necessary but we definitely care about this use case. For example, depending on how we implement entry points, we could just discover the latest interpreter automatically. We might also include a uv tool reinstall command though, it seems generally useful for recovering from problems. Maybe this would be better spelled as uv tool sync [<name>] which would ensure all (or the requested) tool environments match the initial request 🤔

henryiii commented 5 months ago

For uv tool run, would it respect pipx's entrypoint for tools that don't match? (https://github.com/pypa/build/blob/d852035e338bca6a06597cc0501f319d3bf03df4/pyproject.toml#L83-L84) Or would every tool have to add a new entrypoint for uv?

Also, pipx assumes if there's a single console entry point, that that' what pipx run should run, so build would actually work without that now, though IIRC it does print a notice that it's doing that.

Also, pipx reinstall is the only way to update all dependencies. For example, if you've installed twine in the past and now want METADATA 2.3 support, you have to pipx reinstall it since it's actually a dependency that is needed to get that, and pipx only upgrades the main package and anything that it has to. (Also I use reinstall-all after homebrew updates my Python)

ekohilas commented 5 months ago

Forgive me if I've missed something as I haven't had the chance to read everything yet, although

I second @JelleZijlstra's concerns regarding the add and install, as I see it as a main point of confusion for users.

The idea is that uv tool add adds a tool to your project in the same way that uv add will add a dependency to your project. I think once we introduce uv add this will be a bit more intuitive.

I see your point here, and if this was already implemented I would agree. While it's not implemented, then the same case stands for both projects, where having add and install can cause a point of confusion, and could be solved via something like uv dep add or (while it wasn't clear to me what exactly a "project" means here) something like uv tool project install.

I may be missing something though, what are your thoughts?

Excited to see this through!

ekohilas commented 5 months ago

If the tool is not in the project or installed, the latest cached version of the tool will be used. If the tool is not cached, the latest version will be added to the cache.

I really like this, it's really annoying when npx needs to install a package again after having just run it...

The --latest flag can be used to force a check for a newer version (this differs from --no-cache in that the cache can still be used for the tool if it is the latest version and for the tool's dependencies).

If --latest is needed to force check for a newer version, when is the version checked otherwise?

a warning should be provided to the user prompting them to add it to the PATH

Why can't uvx auto update the PATH for the user?

if a tool name is provided, the user will be prompted to confirm an upgrade of all tools.

Would this also update the tools to use the latest version of python installed?

  1. Should we infer the package name from the command name or vice-versa?

I'm not sure I fully understand this. Could you provide an example please?

  1. How should we alleviate confusion around the alias, especially w.r.t. mistakes like uvx install?

For commands, I'd print a warning, and then try to run the install tool. Otherwise, I'd error, and then signal what the correct path should be.

  1. How do users request their project be included with uv tool add?

If I understand correctly, could uv tool add default to adding all projects requested from a particular file, similar to requirements.txt?

  1. How does upgrading work for constrained packages?

Could we try and then warn on failure?

  1. How should we provide re-installation of tools? Does it deserve a dedicated command?

Yes, if the dedicated command would un-install and then re-install, and installing on top hasn't seemed to work on all experiences

  1. Should we allow alternative suffixes to @?

To what benefit?

  1. What's the recommended worfklow for adding dependencies to an already-added or installed tool?

My initial thoughts would be to re-install the tool. I can imagine a case where some tool needs non python packages installed and linked a certain way that a simple addition would not work for.

Thanks for putting this up online, it's written quite well!

zanieb commented 5 months ago

@ekohilas thanks for your comments! (https://github.com/astral-sh/uv/issues/3560#issuecomment-2130784332)

If --latest is needed to force check for a newer version, when is the version checked otherwise?

When the cache entry is missing. Perhaps we'd allow a user to specify that they want to check for the latest version on every invocation but that seems like a rare use-case.

Why can't uvx auto update the PATH for the user?

We cannot mutate the environment of a running shell. We can request that the user add it to their shell startup script though.

Would this also update the tools to use the latest version of python installed?

Unclear. It'd probably make sense unless the user requested a specific version of Python.

I'm not sure I fully understand this. Could you provide an example please?

I don't think this is actually an open question anymore. If you do uvx ruff check we will see the command ruff and install the package ruff. If you do uvx --from ruff-fork ruff check we'll install ruff-check and invoke ruff.

If I understand correctly, could uv tool add default to adding all projects requested from a particular file, similar to requirements.txt?

I'm not sure I follow. The idea with uv tool add is that you would be adding a tool to your pyproject.toml. When running that tool, your project's dependencies would not be available (its environment is isolated). The question is if we should allow the project's dependencies to be included and what the mechanism is e.g. --include-project.

Should we allow alternative suffixes to @?

To what benefit?

pipx feature parity or perhaps you want to do something like foo@stable and foo@beta.

@henryiii (https://github.com/astral-sh/uv/issues/3560#issuecomment-2113460693)

and pipx only upgrades the main package and anything that it has to

This seems weird. I think I'd upgrade all of the packages in the environment

would it respect pipx's entrypoint for tools that don't match? (https://github.com/pypa/build/blob/d852035e338bca6a06597cc0501f319d3bf03df4/pyproject.toml#L83-L84) Or would every tool have to add a new entrypoint for uv?

Oof I did not realize pipx-specific entry points were a thing. I feel like we'd just use normal entry points and not have a special case entry point? I think I need to do some more reading about this to have a strong stance though. Can you share more about the use-case for that?

Regarding the continued concerns about confusion w.r.t. uv tool add vs uv tool install, I'll need to think more about what we can do to address this. We'll probably do "project-level" tools after the user-level installs anyway.

vlad-ivanov-name commented 4 months ago

Question -- will there be a place where tool dependencies are pinned? Normally transitive dependencies of dev-dependencies would be part of a lockfile, but if it's installed globally is there a risk of tool runs not being reproducible between instances of a project when the project references a tool?

zanieb commented 3 months ago

As an update here: uv tool is available and will be stabilized soon. We've dropped project-level tools from scope for now, that part of the design needs more work as people frequently identified it as confusing and there are some complex interactions we need to address.

will there be a place where tool dependencies are pinned? Normally transitive dependencies of dev-dependencies would be part of a lockfile, but if it's installed globally is there a risk of tool runs not being reproducible between instances of a project when the project references a tool?

Yeah project-level tools would need to be included in a version-controlled lockfile.

notatallshaw commented 3 months ago

Will there be a public blog post going through it's features?

While some users might be able to easily migrate to a new tool management system, I think there are many more users who are unsure what and why they would need it. If uv is relatively begining friendly hopefully it can capture some of these users.

zanieb commented 3 months ago

Yes there will be. In the meantime, feel free to provide feedback on the tool guide and concept documentation.

mishamsk commented 3 months ago

@zanieb first, thanks a lot for the link to unreleased docs! that's very revealing, both of the scope and state of the project.

I've been slowly trying to migrate one big project of mine to UV, and we are using poetry dep-groups besides development (docs, test, build) to various things. Most prominently in CI, so we could only install, say, "docs" without the project deps and just run mkdocs or only build (in our case we compile source to C with nuitka).

This is essentially what PEP 735 is about.

From what I understood:

Given all of that, are there any particular plans for PEP 735 functionality? Like for example "we'll wait for PEP 735 to be accepted and will not tackle the case until it is pending", or maybe "we may get to it sooner o despite PEP 735 status but not earlier than X months from now"

For now, I was going to create a bunch of separate requirement files, separate locks and then automate this via some taskfile commands. I do not think that uv lock and uv sync will be a viable option in my setup, but strategic combination of uv pip install -r ... in various combinations will.

PS. If it is a matter of resources - I've been doing some Rust stuff in my spare time lately, I may be able to land a hand here at some point.

T-256 commented 3 months ago

PS. If it is a matter of resources - I've been doing some Rust stuff in my spare time lately, I may be able to land a hand here at some point.

It'd be nice to have a POC work on dependency groups (PEP 735) in uv. by that we can realize how it will improve interface of dev-dependencies (which also includes project dependencies) and build-dependencies (which isolated from project dependencies). I think to achieve that, at first step, we'd need to improve lockfile format to support locking groups and locking inheritances in single file.

zanieb commented 3 months ago

We're not planning on tackling development dependency groups in the first stable release of the project APIs; in part, because PEP 735 is not yet accepted. I've created https://github.com/astral-sh/uv/issues/5632 to track this, let's discuss there instead of polluting the tool discussion.

gaborbernat commented 3 months ago

Any way to install a tool without perfect name match? Something like uv tool install tox-uv tox, so install tox-uv and expose the tox endpoint?

Or install a tool that does not match name wise: uv tool install my.company.namespace.magic providing the magic entry point?

henryiii commented 3 months ago

IMO, I'd recommend supporting the pipx.run entrypoint (with a uv.tool.run or similar entry point taking precedence). Entry points are usually prefixed with the tool that defines them since they are global names, but it doesn't mean that that tool is the only one that can read the entry-point and use it, it's just that tool that defines what that entry point means.

charliermarsh commented 3 months ago

I believe in that case you'd want uvx --with tox-uv tox, which means: "Install tox, install tox-uv in the same environment, then use the tox executable exposed by tox."

uvx --from tox-uv tox would also work, which means: "Install tox-uv, then run the tox executable exposed by tox-uv", but it will warn because tox is actually installed by tox, not tox-uv.

charliermarsh commented 3 months ago

Oh sorry, for install, you'd want uv tool install tox --with tox-uv.

gaborbernat commented 3 months ago

Or install a tool that does not match name wise: uv tool install my.company.namespace.magic providing the magic entry point?

What about this use case?

zanieb commented 3 months ago

Or install a tool that does not match name wise: uv tool install my.company.namespace.magic providing the magic entry point?

What about this use case?

Installing a package installs all of its entry points, that should just work?

gaborbernat commented 3 months ago

Ah, tried again and now it works 🤔 nm me.

zanieb commented 3 months ago

IMO, I'd recommend supporting the pipx.run entrypoint (with a uv.tool.run or similar entry point taking precedence).

I am pretty hesitant to support this without a strong, motivating use-case. I don't think packages should be defining entry points for specific tools (even if other tools can read it). If there's a compelling story here, maybe there should be a PEP to canonicalize it?

matterhorn103 commented 3 months ago

The tool stuff is very nice, but sort of following on from a previous question I had about using tools and the environments/interpreters they are run with, I am still a little unsure about a specific tool example, namely how pyinstaller ought to be run within the uv concept.

(It possibly generalizes to a question of how the design intends tools to be run that rely on seeing/using the venv/dependencies of a project when run to do their job.)

As it is a Python package used via a command-line interface, I would imagine pyinstaller is something that fits into the concept of a "tool". But to work, pyinstaller of course needs to have all the dependencies of the Python program available in the environment it is being run in. Since uvx pyinstaller main.py creates a new environment isolated from the project's, this doesn't work. So what should the approach be? As far as I can tell without testing, all of the below are viable:

  1. Install pyinstaller into the environment and don't run it as a tool but rather using uv run pyinstaller main.py? This feels wrong to me and unfitting to the design, as I said above, it feels like it is a tool. Or would you guys disagree?
  2. As above, but have pyinstaller be a dev dependency, would maybe be neater but has the same counterargument.
  3. uvx --with <dependencies> pyinstaller main.py? This would be quite laborious of course for anything more than a few dependencies.
  4. uv tool install --with <dependencies> pyinstaller, then uvx pyinstaller main.py? This would mean I'd only have to pass the dependencies once, but if I was working on multiple projects that use pyinstaller, wouldn't this give me one massive isolated tool environment with all the dependencies from all the projects?
  5. uvx --with-requirements requirements.txt main.py? I would have thought most uv-managed projects wouldn't have a requirements.txt file.
  6. Some as-yet-unimplemented --with-pyproject option which is similar to the above but works with pyproject.toml?
  7. Some as-yet-unimplemented --with-env option that uses all the same packages as found in the project's .venv? I.e. it doesn't run in the venv per se but the temporary tool venv is set up to mirror it?
  8. Some other option I haven't thought of?

(Something I think would be incredibly cool is if uv offered a uv bundle command that essentially did what pyinstaller does and bundles together everything to afford an executable, so I opened an issue for that, but that seems a bit more high-concept than a solution to the concrete question and would not be relevant more generally.)

zanieb commented 3 months ago

I'd suggest uv run --with pyinstaller -- pyinstaller main.py, which is the intended pattern for running a command with your project installed and an additional dependency.

Alternatively, you could do uvx --with . pyinstaller main.py. I think this is a little more nuanced, as we'd build and install your project into the environment instead of using it as-defined in the workspace — this could actually be beneficial if you're trying to make sure it distributes correctly though?

I think your use-case is pretty clearly addressed in the design though, which says:

A user may opt-in to having their project’s dependencies available in the tool environment.

It's not clear when we'll implement project-level tools though.

matterhorn103 commented 3 months ago

I think your use-case is pretty clearly addressed in the design though, which says:

A user may opt-in to having their project’s dependencies available in the tool environment.

It's not clear when we'll implement project-level tools though.

Ah sorry, you're completely right, I moved over the project-level tools part a bit too fast. I agree that it's clear, consider my question void.

I'd suggest uv run --with pyinstaller -- pyinstaller main.py

Alternatively, you could do uvx --with . pyinstaller main.py.

Thanks for the response, these are nice solutions, especially the nuance of the second. The only thing I would say is that they are a little non-obvious to someone as green as me as they aren't just a direct transfer of the "normal" way to run pyinstaller, so I hope others find this if they ever need it. Since the more general question doesn't make sense anyway, I don't know if you can take my question and your answer out of this thread and put it in its own (closed) issue to make it more findable?

edgarrmondragon commented 2 months ago

--suffix seems to not have been implemented and is the only thing preventing me from migration out of pipx. Should I create an issue?

zanieb commented 2 months ago

@edgarrmondragon feel free to make a targeted issue, yeah.

daviewales commented 2 months ago

A uvx alias will be provided to run tools. Notably, this would be an alias for uv tool run not uv tool — this could be a point of confusion for users transition from pipx but avoids the need to type uvx run. If we're providing an alias, it should be for the most-used command not for the broader uv tool interface. This matches tools in other ecosystems, like npx.

Do you have stats to indicate that uv tool run (or pipx run) is more used than uv tool install or pipx install? Personally, I have used pipx run maybe twice, but I use pipx install regularly. For me, I really like the pipx interface.

mishamsk commented 2 months ago

I have used pipx run maybe twice, but I use pipx install regularly.

I have never used pipe run, but only because pipx puts tool alias into bin folder => there is no need to run pipe run, I just run the tool.

But running the tool should definitely orders of magnitude more often that installing one...

Based on the blog post, I figured uv is not creating a shebanged scripts to allow running tools as if they were binaries on the PATH...so it doesn't seem like a target use case as of today.

zanieb commented 2 months ago

Do you have stats to indicate that uv tool run (or pipx run) is more used than uv tool install or pipx install?

No, but you run things many times and you typically only install them once :) I guess if you have a strong preference for the full interface, alias uvx="uv tool" would get you pretty close?

Based on the blog post, I figured uv is not creating a shebanged scripts to allow running tools as if they were binaries on the PATH...so it doesn't seem like a target use case as of today.

We do put the shebanged scripts in the PATH as pipx does.

henryiii commented 2 months ago

I use pipx run all the time, and use pipx install once in a long while, usually to inject stuff. Which uv’s --with does. I like that it keeps everything up to date (never have to think about what is already installed or what needs updating, regardless of the machine I’m on).

henryiii commented 2 months ago

And, to be clear, since I think there’s some confusion above, they are alternatives of each other, you do not use run on a script you’ve installed (either pipx or uv).

jd-solanki commented 2 months ago

How much clarity it will bring if we rename uv tool to uv global?

# Add a tool to the project
uv global add ruff

# Add a tool with a specific version to the project
uv global add ruff==0.3.0

# Add a tool with an extra dependency
uv global add --with ruff-plugin ruff==0.3.0

# Add a tool to the project that requires the project dependencies at runtime
uv global add pytest --with-project

When I first read docs of rye I was confused about what tool does but I end up realizing it's just global dependency which we can run from anywhere.

If my above understanding is correct then uv global makes more sense.

What do you guys think?

P.S. RFCs are better to discuss in "discussions" so we can reply to other's responses without affecting more inputs (linear comments).

zanieb commented 2 months ago

How much clarity it will bring if we rename uv tool to uv global?

The problem is that you're not able to use that dependency as a Python library, just its CLI. That's why we've avoided terminology like uv global add.

RFCs are better to discuss in "discussions"

We don't have Discussions turned on in this repository and it didn't feel worth it for this one thing. Might be useful in the future.