search

grafana / k6-docs

The k6 documentation website.
https://grafana.com/docs/k6/latest/
84 stars 215 forks source link

Move list of extensions to external file #1640

Open heitortsergent opened 2 months ago

heitortsergent commented 2 months ago

The list of extensions available on the Explore extensions page used to be a JSON file, and we migrated it to be included as part of the versioned files with the Grafana docs move.

Those extensions aren't necessarily versioned the same way that k6 is, so we should move them to a separate place that can represent a single source of truth and then update the documentation to use that file to list them in all versions of the k6 docs.

szkiba commented 1 month ago

A PoC for the external registry: https://github.com/grafana/k6-extension-registry

heitortsergent commented 1 month ago

@szkiba thanks for working on this! From what I understand, we could use this file as the source for the documentation: https://grafana.github.io/k6-extension-registry/extensions.json?

@jdbaldry would it be possible for us to read that file and then feed it into a Markdown file? 🤓

szkiba commented 1 month ago

From what I understand, we could use this file as the source for the documentation: https://grafana.github.io/k6-extension-registry/extensions.json?

Exactly. https://grafana.github.io/k6-extension-registry/registry.json can also be used. extensions.json fits the previous structure a little better (it doesn't have the entry called "k6", there is no "repo" object, it changes less frequently, etc.)

In the meantime, things have changed, I will describe it in a new comment.

I think the extension documentation should be moved outside of the k6 releases. The generation can initially be done with a simple schedule (1-2 times per day). Later, we will be able to solve the change detection and trigger the generation only when a change has occurred.

jdbaldry commented 1 month ago

Nice, yeah we can do this!

There's a couple of ways of the top of my head that you may want to consider:

  1. Sync the data to the website repository and use Hugo templates. This gives us the most flexibility in HTML structure. We would generate the page directly from the data using the Hugo templating language. The downside is that changes in the data wouldn't be immediately available in grafana/docs-base image. I could extend the make docs procedure to support data mounting though and then as long as you have both repositories checked out, you're good to develop in tandem.
  2. Generate a Markdown page source file from the JSON. This introduces a generation step. It is common in other repositories but can be a hurdle for contributors.

In either case, we probably need to write down what's happening somewhere. I'm happy to do that as part of any PR if you tell me where you'd like that documentation to live.

szkiba commented 2 weeks ago

Hi @heitortsergent , @jdbaldry ,

The next milestone of automated registry management has been completed.

The grafana/k6-extension-registry repository contains the extension registry. The source of the registry contains the most important properties, these are supplemented by automation using the GitHub API (e.g. number of stars, versions, etc.).

In the event of a change, other repositories are notified with a repository_dispatch event. Good examples of this are the grafana/k6-extension-catalog and grafana/k6-extension-list repositories. k6-extension-catalog automatically generates JSON catalog files and publishes them using GitHub Pages. k6-extension-list is an example, it automatically generates JSON extension list and markdown extension list in README.md file. It pushes the change to a branch and opens a pull request.

I think k6-docs should work similarly, i.e. a repository_dispatch event would inform about extension registry changes. The workflow downloads the registry from https://grafana.github.io/k6-extension-registry/registry.json and generates what it needs from it, and then publishes the generated extensions documentation.

Will this work for you?

jdbaldry commented 2 weeks ago

Nice!

Yeah I think repository dispatch with that generates changes from the registry JSON seems great to me.

szkiba commented 2 weeks ago

Sounds good.

I added the grafana/k6-docs repository to the list of repositories to be notified. In the event of an extension registry change, a repository_dispatch event will be received, the event_type will be extension-registry-changed. There are no other parameters, the latest version of the extension registry can be downloaded from https://grafana.github.io/k6-extension-registry/registry.json

Let me know if I can help you with anything else regarding the registry.

jdbaldry commented 2 weeks ago

Thank you for your work!

I'll arrange with @heitortsergent when he's back and get something set up to generate the documentation from that registry :)