astral-sh / uv

An extremely fast Python package and project manager, written in Rust.
https://docs.astral.sh/uv
Apache License 2.0
24.84k stars 716 forks source link

Private Projects #8214

Open johnthagen opened 2 weeks ago

johnthagen commented 2 weeks ago

Is it possible to mark a uv project as private such that uv publish is completely disabled? For private code this is a desirable feature.

Something like:

[tool.uv]
private = true

NPM has a similar "private": true:

If you set "private": true in your package.json, then npm will refuse to publish it.

This is a way to prevent accidental publication of private repositories.

In Cargo, this is controlled by the publish flag.

To prevent a package from being published to a registry (like crates.io) by mistake, for instance to keep a package private in a company, you can omit the version field. If you’d like to be more explicit, you can disable publishing:

[package]
# ...
publish = false

For some additional motivation, see a similar feature request for Poetry

zanieb commented 2 weeks ago

This exists already as a Private :: Do Not Upload classifier

zanieb commented 2 weeks ago

This is mentioned over in https://github.com/python-poetry/poetry/issues/1537#issuecomment-1154642727 — we might want to include a private mode that avoids attempting to upload to the registry at all.

cc @konstin

johnthagen commented 2 weeks ago

@zanieb Yeah, I personally feel like the Private :: Do Not Upload classifier is not 100% ideal and having something built in at the package manager level like NPM or Cargo would be more reliable (and also probably easier to teach).

konstin commented 2 weeks ago

I'd love to support something like tool.uv.private = true. The only blocker for this is that currently, uv publish is independent of pyproject.toml: We will upload the files you hand uv publish and if you hand it a wheel, there's no pyproject.toml anymore. We also aren't allowed to set "Private :: Do Not Upload" for you when tool.uv.private = true is set, since we have to respect project.classifiers.

For now, we should start hinting users towards the Private :: Do Not Upload classifier in the docs. It's not the most elegant solution, but it's a reliable way to prevent accidental publishing.

zanieb commented 1 week ago

The only blocker for this is that currently, uv publish is independent of pyproject.toml: We will upload the files you hand uv publish and if you hand it a wheel, there's no pyproject.toml anymore.

I understand that in some contexts we may not know where the wheels come from, but I feel like we should still be able to read project metadata and configuration when publishing artifacts for builds in a project. This seems like a fairly important base capability for uv publish.

We also aren't allowed to set "Private :: Do Not Upload" for you when tool.uv.private = true is set, since we have to respect project.classifiers.

How "not allowed" is this? Couldn't we append this when building the wheel and source dists?

mahyarmirrashed commented 1 week ago

I agree with @zanieb on this. It's fairly important to protect users against shooting themselves in the foot when they accidentally attempt to publish something that is marked as private. As a user, I can provide the following perspective/expected workflow.

  1. If a project is created with uv init, add the private = true by default. The vast majority of projects are not public. Being proactive here brings less harm than the contrary;
  2. If a project has the "Private :: Do Not Upload" classifier, warn the user that it must be removed in order to continue with publishing;
  3. If a project is marked with private = true and the "Private :: Do Not Upload" classifier is not specified, warn the user that they should add it, link to https://pypi.org/classifiers/, and do not publish;
  4. If a project is marked with private = true and the "Private :: Do Not Upload" classifier is specified, warn the user that it is set and therefore, do not publish.

This respects the apparent rule of not setting the classifier automatically while still protecting the user as much as possible. I strongly encourage not adding any --override parameters either. Those usually are a bad idea for users that blindly use them without understanding the consequences and reasons for their errors.

konstin commented 1 week ago

There's two options to implementing private project: METADATA and pyproject.toml.

Metadata

One option is going through the METADATA file and making sure "Private :: Do Not Upload" ends up in the source dist and wheel.

The non-uv solution is adding classifiers = ["Private :: Do Not Upload"] to your pyproject.toml:

[project]
name = "foo-internal"
version = "0.1.0"
classifiers = ["Private :: Do Not Upload"]

This prevents publishing to PyPI.

(There is a separate question that alternative registries such as GitLab have options to host packages both as public and private, but I'm skipping over that here.)

How "not allowed" is this? Couldn't we append this when building the wheel and source dists?

PEP 621 is unambiguous that we must not append to static parts of [project] when building the wheel and source dists, this is required for the ability to read pyproject.toml without a build. There is an escape hatch that requires explicitly declaring specifiers as dynamic, we set the actual classifier on build depending on the value of tool.uv.private, i.e., if we see tool.uv.private = true we transform the metadata on build as if we had classifiers = ["Private :: Do Not Upload"]:

[project]
name = "foo-internal"
version = "0.1.0"
dynamic = ["classifier"]

[tool.uv]
private = true

Through dynamic, we are allowed to emit the following as METADATA:

Metadata-Version: 2.3
Name: foo-internal
Version: 0.1.0
Version: Private :: Do Not Upload

I prefer classifiers = ["Private :: Do Not Upload"] over the above for being self-documenting and more concise.

pyproject.toml

The other option is requiring pyproject.toml to be present.

To check for tool.uv.private, we need to require the pyproject.toml of the (forbidden to be) published packages to be present for all uv publish invocations, for workspace packages this means having all workspace pyproject.toml. If we were to only check if the respective pyproject.toml is present, we would risk ignoring a tool.uv.private.

The other, bigger downside is that this will be ignored when using a standard workflow such as python -m build && twine upload dist/*: Twine, @pypa/gh-action-pypi-publish and other tools will publish this package to pypi with no regards to custom uv configuration, while the classifier protects no matter the tooling mix.

There are other advantages too for requiring pyproject.toml for publish, e.g., reading configuration from it. In the current trade-off, that doesn't tip the scales for me towards switching to a poetry-like uv publish, given that the currently implemented solution is a universally supported self-documenting one-liner:

[project]
name = "foo-internal"
version = "0.1.0"
classifiers = ["Private :: Do Not Upload"]

I agree that cargo and npm have a clearly better state, and would favor the addition of project.publish or project.private to the pyproject.toml spec.

johnthagen commented 1 week ago

The other option is requiring pyproject.toml to be present.

Is there a third option where if pyproject.toml is present, then uv publish inspects it and if private = true is there, it prevents publishing? So normal uv projects will respect it, but if you're using uv publish without a pyproject.toml, then it's back on you to set the classifiers and it works more like twine?

zanieb commented 1 week ago

Yeah I agree, I don't think we need to require it. We could always add a flag that requires we can find it; for people that are concerned about accidentally ignoring their configuration.