Tracker issue: auto-generate reference pages

[ptgott]

Details

This is a long-term tracking issue for creating make targets (or other scripts) to automatically generate our reference documentation.

Auto-generated reference docs are more accurate and easier to maintain than manually written ones, since there's less of a risk of forgetting a configuration resource, field name, possible flag value, etc. They also encourage well-written comments and help text in the source code, which has its own benefits.

However, we would also need to make sure that our auto-generated pages include all of the content we want to preserve in our current, hand-typed pages, e.g., usage examples and caveats.

Here is a table we can use to track issues related to auto-generating our reference docs. The table only includes pages from the current setup/reference/ section of the docs (we'll need to update the table when we reorganize this section). Since protocol-specific sections include their own references, we can consider generating pages for each protocol as well (e.g., see https://github.com/gravitational/teleport/issues/10846).

Page in docs/setup/reference that we could probably generate automatically	Issue	PR	Brief note about viability
setup/reference/config.mdx	https://github.com/gravitational/teleport/issues/21318	The file config is defined as a type in the source, so we could use the ast and astutil packages to extract comments from the struct fields in FileConfig and its fields and use those comments, plus the fields' tags, to generate a reference config file.
setup/reference/cli.mdx	https://github.com/gravitational/teleport/issues/3568, https://github.com/gravitational/teleport/issues/13239	The kingpin package we use for the CLI supports custom output formatting (per #3568).
setup/reference/audit.mdx	https://github.com/gravitational/teleport/issues/5044	#13615	We can use the ast/astutil packages here. Audit events implement an interface, and are emitted by the EmitAuditEvent function.
setup/reference/terraform-provider.mdx	https://github.com/gravitational/teleport/issues/11002	Run protoc-gen-terraform—or import from it as a library—with a docs-specific template.
setup/reference/metrics.mdx	We use the Go Prometheus library to generate metrics, and already include help text when declaring each metric. We can either: - Run a teleport binary so that the Prometheus metrics are registered, then output their texts via working with the *Descs returned by Collector.Describe (i.e., this would be a teleport subcommand that generates metrics docs) - Use Go's AST packages (ast, astutil) to search for instances where we call NewGauge, NewCounter, etc., and extract the name and help text from each options struct passed to the function
setup/reference/resources.mdx	#16948	These are defined as protobuf message types, so we could write a protoc plugin that shares packages with protoc-gen-terraform, which already performs a lot of the necessary work

Category

• Improve Existing

[alexfornuto]

At the risk of bike-shedding, I'd like to propose that we start tackling this issue by auto-generating partial files for each section of the config file based on service type, as well as a partial file for the base parameters required for all service types. Once generated, these partials could be used to break up docs/reference/config into separate pages.

I'd also like to discuss the possibility/feasibility of generating documentation around our schema versioning.

[hugoShaka]

About generating resource reference

During the operator's CRD build process, we already build JSON schema from Teleport protofiles. We're using the Kubernetes' lib, but the whole thing should be able to generate a valid JSON schema AFAIK. We already maintain the conversion logic from our custom types and wrappers to standard props; it would be a shame to develop this again.

About generating config/gotypes reference

I had excellent experiences in the past hijacking Kube tools. apimachinery's source parsing libs have a robust and extensible marker mechanism allowing to attach custom information to types (e.g. which config versions support this field). Elastic's crd rendering tool could be better, but any output format can be supported with a couple of gotemplates.

[ptgott]

During the operator's CRD build process, we already build JSON schema from Teleport protofiles. We're using the Kubernetes' lib, but the whole thing should be able to generate a valid JSON schema AFAIK. We already maintain the conversion logic from our custom types and wrappers to standard props; it would be a shame to develop this again.

I'm going to see if I can extract the JSON schema generation logic from operator/crdgen/schemagen.go into a Go module that we can share between the crdgen plugin and a new plugin for generating the configuration resource reference. That way, the new plugin will only need to execute Go templates with data from the JSON schema.