Ensure we have user documentation on how to consume nightly / beta / RC releases with clusterctl and test framework.

[cahillsf]

Part of Improvement tasks for v1.6 release cycle https://github.com/kubernetes-sigs/cluster-api/issues/9104

• Ensure we have documentation on how to consume beta / RC releases (for users)
• Document how to consume nightly Cluster API releases with clusterctl and with the test framework.

Need to understand the requirements before moving forward with the improvement. cc @g-gaston when you have a chance to expand on this a bit

/area release

[killianmuldoon]

/triage accepted

This will be great for providers - thanks for working on it!

[g-gaston]

TBH, I lack a bit of context here, this precedes me :(

I believe this is to help providers consume the artifacts (so they can do their testing) we produce for beta and RC releases, since these might not be consumable in the same way as "official" releases are. Example: how to use clusterctl from a beta release and make sure it pulls the manifests from that beta release...

@killianmuldoon @sbueringer sorry for the ping but maybe y'all can give a few more examples of instructions that could be useful for provider maintainers? Or maybe I got this completely wrong

[furkatgofurov7]

/assign @cahillsf

[sbueringer]

My idea was to have general documentation on how to use CAPI beta / RC / nightly releases with clusterctl. We produce all of these but probably almost nobody knows how to deploy CAPI with a RC or nightly release.

[furkatgofurov7]

My idea was to have general documentation on how to use CAPI beta / RC / nightly releases with clusterctl. We produce all of these but probably almost nobody knows how to deploy CAPI with a RC or nightly release.

While checking #9104 I noticed we have another very similar task listed there:

Document how to consume nightly Cluster API releases with clusterctl and with the test framework.

which is the same thing, but includes nightly to release to beta/ rc releases and testing with not only clusterctl but also test framework.

@cahillsf I will probably merge these 2 tasks into one if there is no objection from your side since they are very similar😄 (btw, we need to update the issue title + description if the merge happens)

[cahillsf]

Yep all good with me as discussed during the release today @furkatgofurov7 - going to be OOO next week but will try and make time to dig into this when I return

[furkatgofurov7]

/retitle Ensure we have user documentation on how to consume nightly / beta / RC releases with clusterctl and test framework.

[cahillsf]

drafted a PR for a docs update on RC/beta consumption w/clusterctl when you have a chance to review

not sure what the example should look like for consuming nightly releases; clusterctl is pulling the yaml manifests for the tagged versions (betas, rc) from the releases github URL. should i add an example in the developer section then for how to consume the nightly images using Tilt or something along those lines? actually seems like it might be possible to specify the image tag using overrides: https://cluster-api.sigs.k8s.io/clusterctl/configuration#overrides-layer. will take a closer look at this

also could use some help/pointers on the "test framework" piece:

Document how to consume nightly Cluster API releases with clusterctl and with the test framework.

a link to some entrypoints in the code would be appreciated 🙇‍♂️

cc @furkatgofurov7 @sbueringer

[sbueringer]

@cahillsf Sorry for the late response. We are publishing the nightly manifests to google cloud storage and they can be referenced like this: https://github.com/kubernetes-sigs/cluster-api-provider-aws/blob/720b586403edb0517f9c818778421900e4b671a8/test/e2e/data/e2e_eks_conf.yaml#L74 (the file names should match the ones in our release attachments)

(a more recent URL: https://storage.googleapis.com/artifacts.k8s-staging-cluster-api.appspot.com/components/nightly_main_20231122/core-components.yaml)

I'm not sure if it's possible to also consume those manifests with clusterctl. The nice thing is that those manifests are actually the nightly ones and contain the right nightly images (while only overwriting the images doesn't give you the right manifests)

EDIT: Ah probably it's possible like described here: https://cluster-api.sigs.k8s.io/clusterctl/configuration#provider-repositories

Basically just overwrite an existing provider with a URL that points to nightly manifests (not 100% sure if clusterctl accepts these URLs)

[cahillsf]

@sbueringer np thanks for taking a look. reviewed your suggestions and you are right, this piece currently is not supported:

I'm not sure if it's possible to also consume those manifests with clusterctl.

Specifying the google storage URL as an override for the provider repo kicks back the following: Error: failed to get provider components for the "cluster-api" provider: failed to get repository client for the CoreProvider with name cluster-api: invalid provider url. Only GitHub and GitLab are supported for "https" schema

Here's the line in the source code where this is thrown: https://github.com/kubernetes-sigs/cluster-api/blob/2d58e1c6f5efc509f185544a455e5125374215df/cmd/clusterctl/client/repository/client.go#L197

If I'm understanding correctly, I believe the solution is to implement a repository_googlestorage.go (or something like that) here: https://github.com/kubernetes-sigs/cluster-api/tree/main/cmd/clusterctl/client/repository and then accept URLs like: https://storage.googleapis.

Does that sound right to you?

[sbueringer]

Sounds correct but I'm not an expert on all things clusterctl 😃

(@fabriziopandini )

[cahillsf]

👋 @fabriziopandini when you have a chance, can you confirm if this is the correct approach? https://github.com/kubernetes-sigs/cluster-api/issues/9604#issuecomment-1859213714

(TLDR trying to allow clusterctl to consume the nightly release manifests from URLs like: https://storage.googleapis.com/artifacts.k8s-staging-cluster-api.appspot.com/components/nightly_main_20231122/core-components.yaml)