Automate CRD documentation

[mmitoraj]

Description

Kyma documentation in the Technical Reference section focuses on Kyma Custom Resources. The documents include tables listing all the possible parameters of a given resource together with their descriptions. As the parameters and CRDs change, the documentation may easily become out-of-date and difficult to maintain. Huskies came up with the idea to auto-generate the content of the tables. For example, see the auto-generated table for the Telemetry - LogPipeline CR parameters CR. The solution could be improved, adjusted, and implemented for all CRs descriptions.

Steps:

1. [x] Ask @a-thaler to explain the mechanism so that other teams could confirm if the solution fits their use cases and requirements
2. [x] Discuss with @kyma-project/technical-writers if the solution meets Kyma documentation requirements
3. [x] Contact POs to consult if auto-generating the tables' content with their teams' CRDs description is possible
4. [x] Improve and adjust the mechanism if needed
5. [ ] Automate the docu creation

Area

Area	Custom resource
Application Connectivity	Application, CompassConnection
API Gateway	APIRule
Eventing	Subscription, EventingBackend
Istio	Istio
Serverless	Function, GitRepository
Telemetry	LogPipeline, LogParser, TracePipeline

Reasons

Up-to-date documentation.

Assignees

@mmitoraj @a-thaler

Related issues

17421 #17380

[a-thaler]

• every domain need to provide good and complete descriptions on the CRD attributes
• prepare your MD files by removing the relevant tables and replacing them with the placeholders as described here
• add new mappings for your CRDs to that Makefile, defining to map which CRD to which file
• when providing a new CRD version to the kyma folder, execute first the "make generate" command
• a prow job will remind you if you forgot the previous step as it will execute the same make target and check the git status afterwards

[NHingerl]

I'm absolutely in favor of the idea, :+1: :+1: :+1:

[mmitoraj]

Dear All,

To drive the topic forward, we would need your and your teams' engagement at that point. I created a table listing Kyma CRDs. Please:

• check if documentation for your CRDs can be automated as in the example provided in the issue description
• add your ideas on top so that the process works for your use case
• estimate your team's effort for the task and create an issue for it

Hints: The parameters description is mostly there in the documentation; however, it needs to be reviewed and moved to the specification files. CAUTION: The CRDs may require updating.

Please ask any questions you may have.

Kyma CRDs per teams:

CRD	Team	Contact Person
apirules.gateway.kyma-project.io	Goats	@strekm
istios.operator.kyma-project.io	Goats	@strekm
applications.applicationconnector.kyma-project.io	Framefrogs	@Disper
compassconnections.compass.kyma-project.io	Framefrogs	@Disper
functions.serverless.kyma-project.io	Otters	@kwiatekus
gitrepositories.serverless.kyma-project.io	Otters	@kwiatekus
eventingbackends.eventing.kyma-project.io	Skydiving Tunas	@k15r
subscriptions.eventing.kyma-project.io	Skydiving Tunas	@k15r
tracepipelines.telemetry.kyma-project.io	Huskies	@a-thaler
logparsers.telemetry.kyma-project.io	Huskies	@a-thaler
logpipelines.telemetry.kyma-project.io	Huskies	@a-thaler

[a-thaler]

I'm all in, let me know when we should do what

[k15r]

Great idea, we're in

[mmitoraj]

AIs from today's synch:

• Create a ticket for each team to provide/update their CRDs attributes description [@strekm, @k15r @kwiatekus @VOID404, @a-thaler]
• Prepare the .md files by removing existing tables and adding placeholders once the descriptions are ready and the makefile is updated [@mmitoraj]
• Consider improvements, such as adding the Mandatory column and the possibility to hide certain fields
• Schedule a meeting in 2 weeks' time [@mmitoraj]

[mmitoraj]

11.05.2023 synch:

• New features and improvements added to the solution - 1️⃣ Information if a parameter is required added in parentheses next to the param name. 2️⃣ The long parameter names can break so the Parameter name column and the whole table are not that wide and ugly anymore. 3️⃣ Type column added. Thank you @a-thaler and @k15r! ❤️ For more details check the below PRs.
  • https://github.com/kyma-project/kyma/pull/17421
  • https://github.com/kyma-project/kyma/pull/17467
• 🐸 Framefrogs updated their CompassConnection CR description. AppllicationConnection CRD - update in progress [@VOID404]
• 🦦 Otters updated Function CRD, GitRepo CR is deprecated. @anoipm working on mapping the CRD files to the makefile in this PR. [@kwiatekus]
• Huskies are done with their changes. [@a-thaler] - However, please bear in mind that there are still some empty places in the description columns in your documentation ❗
• Skydiving Tunas - adding the descriptions in progress #17450
• 🐐 Goats - @strekm please share your update at your earliest convenience

[strekm]

@a-thaler @k15r so far CRD is fetched from kyma repo. i'm assuming that for modularisation it will be possible to fetch it from different repo, right?

[a-thaler]

There will be no central website repo where the content gets copied to, instead the idea is that the website gets rendered directly with the content from the different repos. So you would need to render/generate the .md files in your repo. So you would need a custom automation in your repo. Not sure yet how we want to share the tooling for that, maybe we can place it centrally in the kyma repo and you load it as part of your module automation from remote?

[k15r]

With essentially all code moving out of the kyma repository it might make sense to have a dedicated repository for the crd-documentation tooling. As the generation itself is a single line command which could be easily forged into a reusable tekton pipeline step or also a github action

[mmitoraj]

@VOID404 It seems like the Application CR documentation is the last one to automate. Could you let me know what is needed to complete the task? Do you need any assistance with that one? Thank you in advance.

[Disper]

Hi @mmitoraj, we don't need assistance and @kyma-project/framefrog part is already pretty advanced, however we have two high risk topics on the managed part on our plates right now and we have to fully focus on them. Right now I believe that we should be able to jump back to this topic somewhere in June.

[mmitoraj]

@Disper Hi Michał, can we expect the task to be taken up soon?

[Disper]

Hi @mmitoraj, I'll make sure that we will pick it up next 2-week sprint that starts on Thursday.

[Disper]

@mmitoraj, we've discussed the topic today and while there is a chance that will be finishing that in the upcoming sprint, we can't guarantee it due to some internal incidents and one external one that appeared and we have to focus on.

As team @kyma-project/framefrog is the only one lagging behind, should we maybe take over the assignment of this task and close it when it will be done?

[mmitoraj]

@Disper, yes, team Framefrog is the only one left. That is a good idea for you to take over the assignment. Let me know if you need any help. Thank you/

[kyma-bot]

This issue or PR has been automatically marked as stale due to the lack of recent activity. Thank you for your contributions.

This bot triages issues and PRs according to the following rules:

• After 60d of inactivity, lifecycle/stale is applied
• After 7d of inactivity since lifecycle/stale was applied, the issue is closed

You can:

• Mark this issue or PR as fresh with /remove-lifecycle stale
• Close this issue or PR with /close

If you think that I work incorrectly, kindly raise an issue with the problem.

/lifecycle stale

[kyma-bot]

This issue or PR has been automatically marked as stale due to the lack of recent activity. Thank you for your contributions.

This bot triages issues and PRs according to the following rules:

• After 60d of inactivity, lifecycle/stale is applied
• After 7d of inactivity since lifecycle/stale was applied, the issue is closed

You can:

• Mark this issue or PR as fresh with /remove-lifecycle stale
• Close this issue or PR with /close

If you think that I work incorrectly, kindly raise an issue with the problem.

/lifecycle stale

[kyma-bot]

This issue or PR has been automatically marked as stale due to the lack of recent activity. Thank you for your contributions.

This bot triages issues and PRs according to the following rules:

• After 60d of inactivity, lifecycle/stale is applied
• After 7d of inactivity since lifecycle/stale was applied, the issue is closed

You can:

• Mark this issue or PR as fresh with /remove-lifecycle stale
• Close this issue or PR with /close

If you think that I work incorrectly, kindly raise an issue with the problem.

/lifecycle stale

[a-thaler]

Leftover action is in the application connector which is handled individual in https://github.com/kyma-project/application-connector-manager/issues/155. Will close this epic already