k0sproject / k0smotron

k0smotron
https://docs.k0smotron.io/
Other
507 stars 47 forks source link

Autopilot plan is not deleted after running a successful upgrade #816

Open dvirassamy opened 3 days ago

dvirassamy commented 3 days ago

What happened?

Running successive upgrades on child clusters fails.

Involved Areas

Cluster API integration

What did you expect to happen?

  1. Multiple upgrades of the child cluster should work.
  2. Autopilot should keep history of the upgrade plans

Steps to reproduce

Step 1. Apply manifests with new version definition. The change would trigger an upgrade command

k0smotron successfully runs the k0s upgrade operations.

Step 2 Apply again manifests with new version definition. The change would trigger an upgrade command

k0smotron fails with the following error:

2024-11-18T13:35:13Z    ERROR   Reconciler error        {"controller": "k0scontrolplane", "controllerGroup": "[controlplane.cluster.x-k8s.io](http://controlplane.cluster.x-k8s.io/)", "controllerKind": "K0sControlPlane", "K0sControlPlane": {"name":"child-02","namespace":"child-02"}, "namespace": "child-02", "name": "child-02", "reconcileID": "213dd124-c6ff-4db5-9687-5dbfb6b5d9e0", "error": "error creating autopilot plan: [plans.autopilot.k0sproject.io](http://plans.autopilot.k0sproject.io/) \"autopilot\" already exists"}
[sigs.k8s.io/controller-runtime/pkg/internal/controller](http://sigs.k8s.io/controller-runtime/pkg/internal/controller).(*Controller).reconcileHandler
        /go/pkg/mod/[sigs.k8s.io/controller-runtime@v0.16.5/pkg/internal/controller/controller.go:329](http://sigs.k8s.io/controller-runtime@v0.16.5/pkg/internal/controller/controller.go:329)
[sigs.k8s.io/controller-runtime/pkg/internal/controller](http://sigs.k8s.io/controller-runtime/pkg/internal/controller).(*Controller).processNextWorkItem
        /go/pkg/mod/[sigs.k8s.io/controller-runtime@v0.16.5/pkg/internal/controller/controller.go:266](http://sigs.k8s.io/controller-runtime@v0.16.5/pkg/internal/controller/controller.go:266)
[sigs.k8s.io/controller-runtime/pkg/internal/controller](http://sigs.k8s.io/controller-runtime/pkg/internal/controller).(*Controller).Start.func2.2
        /go/pkg/mod/[sigs.k8s.io/controller-runtime@v0.16.5/pkg/internal/controller/controller.go:227](http://sigs.k8s.io/controller-runtime@v0.16.5/pkg/internal/controller/controller.go:227)

As Autopilot cannot handle two plans with the same name, the upgrade operation fails.

Knowing this limitation of Autopilot, k0smotron should create unique named upgrade plans with unique IDs. This is important to keep the plans history for audit purposes.

k0smotron version

1.2.0

k0s version

*

Anything else we need to know?

No response

jnummelin commented 2 days ago

@makhov I think this boils down to this line: https://github.com/k0sproject/k0smotron/blob/main/internal/controller/controlplane/helper.go#L256-L257

So we do POST request, shouldn't we always do merge, as in like kubectl apply ...