server: rework the layout of HTTP APIs

[knz]

This issue is an "epic" that describes what we are aiming for through the 21.1 cycle and beyond.

Summary: in v21.1 we are going to:

• sunset the /_admin/v1, and /_status URL prefixes for consumption by end-users. In particular /_status will be eventually removed e.g. in v21.2.
• instead, introduce a new /api/v2 URL prefix under which all public services will be available. From then on, all API additions for end-users will go under /api and follow our general API versioning rules. It will be possible for multiple versions to co-exist side-by-side to offer forward and backward compatibility to clients.
• The Admin UI will continue to get all the information it needs under /_admin which will not be versioned and will evolve in lock-step with the UI code, for simplicity by CockroachDB developers. There will be no guarantees offered to external users about the behavior of /_admin. We will also migrate anything under /_status and /debug used by the admin UI to live under /_admin.
• if a service is useful to both the admin UI and end-users, then it will have two URLs: one under /api and one under /_admin. Only the former will be public and documented.
• /debug will continue to be a special "anything goes" so that CockroachDB developers can quickly add or change troubleshooting endpoints there. However, any debug endpoint useful for the admin UI will also have an entry point under /_admin and any debug endpoint with value to end-users will need to be "promoted" under the /api tree.
• /health will also continue to be special and will not change URLs nor behavior in this cycle. However, we're going to attempt to pull it out of mandatory TLS, so that non-TLS HTTP clients can check health too. (This is a separate project)

Internal changes

We acknowledge that all our HTTP APIs today are also not used internally by RPC clients inside CockroachDB; and conversely all RPC services are otherwise not used by HTTP clients. Therefore:

• we are going to bring all the current server methods for HTTP endpoints under a HTTP server directly.
• we are going to disconnect the RPC services from the gRPC-HTTP gateway.

This will greatly reduce the complexity of the server code, and also ensure that our RPC and HTTP authentication mechanisms are clearly separate.

Then at a later phase (see separate issue TBD) connect the SQL layer to the (new) HTTP methods via the new HTTP-over-SQL gateway (new project/component).

Possible strategy (needs to be discussed / refined further)

1. add a "native" Go http method dispatcher to the server code.
2. exercise step (1) by seeding the /api/v1 tree with just a few of the APIs we already know are popular and need to be provided:
   • list of nodes in cluster
   • list of hot ranges
3. concurrently with steps 1 and 2, adapt the admin UI client code to only query endpoints under /_admin (i.e. bring all uses of /_status under /_admin, and add URL aliases accordingly
4. concurrently with steps 1-3, initiate a documentation project with docs to explain this transition
5. fix issue #54022 for the few new APIs in step (2) by creating example pagination and some Go code skeletons that all future API additions can follow / reuse for pagination.
6. exercise (5) by incrementally adding more complex APIs to check that the pagination is robust. Starting with:
   • list of jobs
   • list of log entries
7. concurrently with (3)-(6) work on the new HTTP-over-SQL gateway so that SQL code can also query the public APIs.

Related wiki pages:

• API Design guidelines (needs to be made public eventually)
• HTTP-over-SQL gateway (needs to be made public eventually)

Related github issues:

• server, ui: Merge Admin and Status endpoints #24989
• server: introduce versioning and pagination in the HTTP APIs #54022
• server: the HotRangesResponse contains raw RangeDescriptors with sensitive data #53212

Jira issue: CRDB-3609

[blathers-crl[bot]]

Hi @knz, I've guessed the C-ategory of your issue and suitably labeled it. Please re-label if inaccurate.

While you're here, please consider adding an A- label to help keep our repository tidy.

_{:owl: Hoot! I am a Blathers, a bot for CockroachDB. My owner is otan.}

[knz]

@bdarnell this issue reflects what we talked about last week.

@itsbilal as you have expressed interest in this, I'd ask you to review the issue, check whether the overall direction makes sense, and whether you have any improvements to the proposed plan. If it all makes sense, then I'd encourage you to file more fined grained issues that identify the individual steps we'd need to carry out, and for each of them describe the specific direct outcomes we're expecting (so that it's clear when we can close the issue).

[knz]

@taroface I encourage you too to look at the "summary" at the top of this issue, as it follows up on the initiatives in the Dev Interfaces team and suggests how we're going to move forward. If you have any suggestions about UX for end-users, we'll take them too.

[github-actions[bot]]

We have marked this issue as stale because it has been inactive for 18 months. If this issue is still relevant, removing the stale label or adding a comment will keep it active. Otherwise, we'll close it in 10 days to keep the issue queue tidy. Thank you for your contribution to CockroachDB!