[META] Determine API design for performing maintainer actions

[di]

What's the problem this feature will solve? We've had several requests for an API that would let users perform various maintainer actions only possible via the web UI right now:

• Deleting releases/files (https://github.com/pypi/warehouse/issues/12934)
• Yanking releases
• Generating API tokens (https://github.com/pypi/warehouse/issues/6396)
• Regenerating/invalidating API tokens (https://github.com/pypi/warehouse/issues/12765)
• Configuring trusted publishers (https://github.com/pypi/pypi-oidc-private-beta-community/issues/34) (https://github.com/pypi/warehouse/issues/14456)

Describe the solution you'd like Before attempting to implement any of them, we should figure out how we would generally want to expose these types of APIs to maintainers, the API format, authentication mechanism, etc.

[takluyver]

API format: some sort of REST + JSON, though I guess this is now about as generic as saying 'implement it using computers' :wink: . E.g. for deletion, it would be nice if each file and release had an API url, and you used the HTTP DELETE verb on that URL to delete it. Obviously there are a whole lot of details to work out here, but I imagine those will mostly be hammered out for each specific piece of API.

Authentication: I imagine some flavour of Oauth is the obvious answer? Please ensure there's a clear story for people authenticating scripts etc. from their own machine, though. Too many APIs with Oauth assume that you'll only ever sign in through another server which can have a 'client secret', and force users to 'create an application' and copy and paste stuff around just to do something simple.

[webknjaz]

@jaraco ^

[woodruffw]

Just doing some sketching:

The current trusted publishing APIs live under /_/, which (IMO) is a fitting place for PyPI-specific APIs as well. In terms of routes and verbiage, it probably makes sense to roughly mirror the HTML routes/views where possible while being RESTful, e.g.:

• /_/api/manage/project/release: DELETE for deletion, PATCH for yanking (?)
• /_/api/manage/token: DELETE for "burning" a token
  • The equivalent routes here are under /manage/account/token, but with trusted publishing a token is no longer necessary associated with an account.

For additive operations (like adding new trusted publishers, creating new API tokens, etc.) we should think carefully about the permission model: if it's too loose, we might end up encouraging users to create "god" tokens for API interactions that they end up sharing/using somewhat haphazardly. For example, we should probably be very careful about how (if) we allow an API to mint more API tokens, and ensure that the majority of users are funneled to trusted publishing for that kind of use case instead.

CC @tnytown for visibility

[woodruffw]

Another thing to consider: we currently offer "user-scoped" API tokens, which just means that the token is valid for every project the user has. We probably don't want to inadvertently broaden the scope of those "user" tokens to all user management tasks as well, so we should probably think about changing the name of that Macaroon caveat 🙂

[takluyver]

I guess authentication is one of the first pieces to figure out. I see a couple of options for how this will work from a user perspective:

1. You enter your PyPI username, password & 2FA code in the app (e.g. at terminal prompts), and it sends these to PyPI, exchanging them for a token it can use to authenticate later requests.
2. The app opens a web browser, or gives you a URL to open, where you log in to PyPI and grant it access.

I think 1. is more flexible - no need for a browser - and a nicer user experience - no need for a disorienting switch into a browser and back. But 2. is probably more secure - you have a chance to check what permissions you're granting, and it's harder for a clumsily coded app to save your password. 2. also means apps don't need to deal with 2FA security keys.

What do you get once authenticated? Would the token let you upload packages directly, or would you use the API token to create an upload token? Presumably the token you get would have some expiry date? But upload tokens are perpetual by default - does anyone want to come up with advice for tools on which sort of token to store?

[woodruffw]

I might have misunderstood, but I thought the assumption here was that these APIs would be limited to API tokens only.

In other words: the authentication flow here would be to create an appropriately scoped API token through the PyPI web interface, and then authenticate to the API with that token. That in turn would allow us to reuse all of the existing scaffolding we have around API token creation, without having to solve problems like doing TOTP challenges in the user's terminal.

[dstufft]

I believe almost all of the APIs would use API Tokens yes.

One of the things people have asked for is an API to generate tokens though, which would involve some kind of login flow.

[takluyver]

Yup, I'd like to have a better way to upload projects than manually creating, copying and pasting a token for each project (besides storing a user-scoped token, which is basically as powerful as my password).

[alanhamlett]

I'd like Read-Only API Tokens for user accounts (not projects), or the equivalent OAuth scope'd token if a PyPI OAuth provider is implemented in the future.

[dubov94]

For an API-based reimplementation of our PyPI management tool the we'd need:

• an authentication mechanism;
• endpoints for invitation management;
• a way of listing projects for an authenticated account;
• a way of listing the contributors (including their roles and 2FA status) to those projects.

I suppose some of the required functionality could become obsolete once https://blog.pypi.org/posts/2023-04-23-introducing-pypi-organizations/ is fully rolled out for us.