[Fleet] Automate the generation of Fleet's OpenAPI spec through `@kbn/config-schema`

[kpollich]

Today, Fleet's OpenAPI spec file is generated and maintained through a manual process documented in the README file in x-pack/fleet/plugins/fleet/common/openapi/.

Kibana as a whole is moving towards support for fully-automated OpenAPI specs for plugin APIs, as tracked here: https://github.com/elastic/kibana/issues/180056. Fleet's manual process should be replaced by a fully automated one in which the OAS spec is inferred from metadata on our server-side routes and request/response contracts defined via @kbn/config-schema.

We had previously done some investigation into whether @kbn/openapi-generator was ready to meet Fleet's needs and the answer at the time was "no". However, with a few months of progress since then, we should start the process of moving towards automating our schema generation. Manually generating and maintaining our OpenAPI spec is labor intensive and error prone, so shedding this friction will speed up our development process on the team.

Here is an example of the /api/status Kibana API's schema written with @kbn/config-schema that results in an OpenAPI spec here:

https://github.com/elastic/kibana/blob/3efa595a9ad52e63324fc3009ab7c9fb52174eed/oas_docs/bundle.json#L451-L525

What we'll actually need to do to accomplish this is to replace each entry we have today in Fleet's OpenAPI spec files with a more advanced set of types created with @kbn/config-schema alongside each route we register.

An example of this for the /api/fleet/setup API might look like this:

diff --git a/x-pack/plugins/fleet/server/routes/setup/index.ts b/x-pack/plugins/fleet/server/routes/setup/index.ts
index f09ff70e145..054fb7a2754 100644
--- a/x-pack/plugins/fleet/server/routes/setup/index.ts
+++ b/x-pack/plugins/fleet/server/routes/setup/index.ts
@@ -13,11 +13,13 @@ import { API_VERSIONS } from '../../../common/constants';
 import type { FleetConfigType } from '../../../common/types';

 import { getFleetStatusHandler, fleetSetupHandler } from './handlers';
+import { schema } from '@kbn/config-schema';

 export const registerFleetSetupRoute = (router: FleetAuthzRouter) => {
   router.versioned
     .post({
       path: SETUP_API_ROUTE,
+      description: 'Initiate Fleet setup',
       fleetAuthz: {
         fleet: { setup: true },
       },
@@ -25,7 +27,51 @@ export const registerFleetSetupRoute = (router: FleetAuthzRouter) => {
     .addVersion(
       {
         version: API_VERSIONS.public.v1,
-        validate: false,
+        validate: {
+          request: {},
+          response: {
+            200: {
+              body: () =>
+                schema.object(
+                  {
+                    isInitialized: schema.boolean(),
+                    nonFatalErrors: schema.object({
+                      name: schema.string(),
+                      message: schema.string(),
+                    }),
+                  },
+                  {
+                    meta: {
+                      description:
+                        "A summary of the result of Fleet's `setup` lifecycle. If `isInitialized` is true, Fleet is ready to accept agent enrollment. `nonFatalErrors` may include useful insight into non-blocking issues with Fleet's health.",
+                    },
+                  }
+                ),
+            },
+            400: {
+              body: () =>
+                schema.object(
+                  {
+                    statusCode: schema.number(),
+                    error: schema.string(),
+                    message: schema.string(),
+                  },
+                  {
+                    meta: { description: 'Generic error' },
+                  }
+                ),
+            },
+            500: {
+              body: () =>
+                schema.object(
+                  {
+                    message: schema.string(),
+                  },
+                  { meta: { description: 'Internal server error' } }
+                ),
+            },
+          },
+        },
       },
       fleetSetupHandler
     );

Each documented status code, response body, query parameter, etc can be documented directly alongside the route definition. We'd need to take what we have today in the various OpenAPI spec files and translate them to this approach for every Fleet API endpoint. This will be no small undertaking, but it will greatly improve our ability to create or iterate on API endpoints It will also greatly empower our docs team to generate better API documentation.

[elasticmachine]

Pinging @elastic/fleet (Team:Fleet)