search

kubernetes-sigs / cluster-api-provider-ibmcloud

Cluster API Provider for IBM Cloud
https://cluster-api-ibmcloud.sigs.k8s.io
Apache License 2.0
62 stars 81 forks source link

Bootstrap cluster with ignition #843

Open mkumatag opened 2 years ago

mkumatag commented 2 years ago

/kind feature /area provider/ibmcloud

Describe the solution you'd like https://cluster-api.sigs.k8s.io/tasks/experimental-features/ignition.html

Anything else you would like to add: [Miscellaneous information that will assist in solving the issue.]

k8s-triage-robot commented 1 year ago

The Kubernetes project currently lacks enough contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

You can:

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

k8s-triage-robot commented 1 year ago

The Kubernetes project currently lacks enough contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

You can:

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

mkumatag commented 1 year ago

/remove-lifecycle stale

mkumatag commented 1 year ago

Fedora CoreOS is still pending for ppc64le, will explore this task once builds are available for the testing.

Prajyot-Parab commented 12 months ago

/assign @Prajyot-Parab Update - Explore on booting Fedora CoreOS with ignition for CAPIBM PowerVS K8s cluster.

@mkumatag can you please add the reference links/urls for the Fedora CoreOS (ppc64le) builds.

yussufsh commented 12 months ago

@Prajyot-Parab https://fedoraproject.org/coreos/download/?stream=stable&arch=ppc64le#download_section you can download ppc64le stable images from here.

mkumatag commented 10 months ago

@Prajyot-Parab can you document the initial investigation you have done and move this work item to next release.

Prajyot-Parab commented 10 months ago

/milestone Next

Prajyot-Parab commented 9 months ago

The default configuration engine for bootstrapping workload cluster machines is cloud-init. Ignition is an alternative engine used by Linux distributions such as Flatcar Container Linux and Fedora CoreOS and therefore should be used when choosing an Ignition-based distribution as the underlying OS for workload clusters.

Note This initial implementation uses Ignition v2 and was tested with Flatcar Container Linux only. Future releases are expected to add Ignition v3 support and cover more Linux distributions.

@mkumatag as per my initial understanding we will need to build the flatcar stable image for powervs, once we have that we can try the already tested workflow using it.

Not sure how this is going to work with Fedora CoreOS-

  1. We will need some modifications on top of it to make it CAPIBM ready right? If yes, will this be somehow done as part of image builder process or it will be dealt with externally?

@mkumatag need your guidance on this one.

mkumatag commented 9 months ago

Flatcar image is not available for the power, let's explore the experiment with the fedoracore image directly and see what kind of errors we are hitting while running the flow?!

Prajyot-Parab commented 8 months ago

Summary - We explored with FerdoraCoreOS 39 for Power

  1. The version of ignition inside image is 2.17.0 => 3.x.x spec
  2. The created infra machine was able to fetch the ignition from mentioned URL.
  3. The issue is with config version mismatch while booting above machine via ignition. This is result of ignition file being generated using older spec 2.3.0.
  4. Ignition file is generated while upstream CAPI (which in-turn uses flatcar package), which supports only 2.3.0 and no 3.x.x specs.

This will possibly need more exploration and upstream CAPI contributions to add support for spec 3.x.x ignition versions.

Prajyot-Parab commented 8 months ago

@Karthik-K-N feel free to add more info, if i missed anything.

k8s-triage-robot commented 5 months ago

The Kubernetes project currently lacks enough contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

You can:

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

mkumatag commented 5 months ago

/remove-lifecycle stale

Karthik-K-N commented 5 months ago

WIP PR is submitted https://github.com/kubernetes-sigs/cluster-api-provider-ibmcloud/pull/1687, Need to complete it.

k8s-triage-robot commented 2 months ago

The Kubernetes project currently lacks enough contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

You can:

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

Karthik-K-N commented 1 month ago

Currently CAPIBM controllers supports booting machine with ignition with 3.x version which is extensively used in Openshift clusters where the ignition is generated by installer and which has version 3.x.

As a standalone CAPIBM cluster we are yet to support ignition because the CAPI has support for generating ignition with only 2.3 version.

Possible goals?

  1. Should we make changes in CAPIBM to support very old ignition 2.3.X? (is it worth to make changes as ignition 3.x flow will always be tested in IPI flow)

Challenges

  1. Many other providers use flatcar which supports ignition and image builder repo also has support for building flatcar image for CAPI, Unfortunately it does not support ppc64le arch.

Alternatives

  1. Use very old rchcos image which has igntion 2.3.x support, make changes in image builder to build CAPI compatible image for this and try with CAPIBM.
  2. Identify and use converter package which converts ignition from 2.3.x to 3.x

Upstream CAPI issues where its currently in planning state to support ignition 3.x version.

  1. https://github.com/kubernetes-sigs/cluster-api/issues/9157
  2. https://github.com/kubernetes-sigs/cluster-api/issues/5294
mkumatag commented 1 month ago

@Karthik-K-N can we list down the pro and cons for both the approaches mentioned and debate from there?

Amulyam24 commented 4 weeks ago

/remove-lifecycle stale