search

ionos-cloud / cluster-api-provider-proxmox

Cluster API Provider for Proxmox VE (CAPMOX)
Apache License 2.0
150 stars 21 forks source link

Capmox 0.5.0 unable to access proxmox-credentials secret #245

Open mkamsikad2 opened 2 weeks ago

mkamsikad2 commented 2 weeks ago

What steps did you take and what happened: I upgraded from Capmox 0.4.0 to 0.5.0

clusterctl upgrade apply --core cluster-api:v1.7.2 --bootstrap kubeadm:v1.7.2 --control-plane kubeadm:v1.7.2 --ipam in-cluster:v0.1.0 --infrastructure proxmox:v0.5.0

Following the upgrade the Capmox controller logs the following errors:

E0619 13:51:44.068023 1 controller.go:329] "Reconciler error" err="error getting infra provider cluster or control plane object: No credentials found, ProxmoxCluster missing credentialsRef" controller="proxmoxmachine" controllerGroup="infrastructure.cluster.x-k8s.io" controllerKind="ProxmoxMachine" ProxmoxMachine="default/capi-management-v2-control-plane-qjwxf" namespace="default" name="capi-management-v2-control-plane-qjwxf" reconcileID="2ae9413d-cb8e-4425-9860-e870d84adb5a"

I then added created the required secret and added the secret in ProxmoxCluster.spec.credentialsRef.name

apiVersion: v1
stringData:
  secret: ${PROXMOX_SECRET}
  token: ${PROXMOX_TOKEN}
  url: ${PROXMOX_URL}
kind: Secret
metadata:
  name: "${CLUSTER_NAME}-proxmox-credentials"
  labels:
    platform.ionos.com/secret-type: "proxmox-credentials"

The capmox provider cannot then find the secret and is unable to communicate with proxmox:

E0619 13:51:43.567318       1 controller.go:329] "Reconciler error" err="error getting infra provider cluster or control plane object: Unable to initialize ProxmoxClient: failed to get credentials secret: Secret \"mk1-busi-cl-proxmox-credentials\" not found" controller="proxmoxmachine" controllerGroup="infrastructure.cluster.x-k8s.io" controllerKind="ProxmoxMachine" ProxmoxMachine="default/mk1-busi-cl-worker-pzvch" namespace="default" name="mk1-busi-cl-worker-pzvch" reconcileID="de7a6641-3c01-443a-8c5b-02f2cd62797a"
    failed to create scope: Secret "mk1-busi-cl-proxmox-credentials" not found
[14:02:29] k8sadmin@image-k get secret mk1-busi-cl-proxmox-credentials
NAME                              TYPE     DATA   AGE
mk1-busi-cl-proxmox-credentials   Opaque   3      26m
E0619 13:57:18.765794       1 controller.go:329] "Reconciler error" err=<
    failed to create scope: Secret "mk1-busi-cl-proxmox-credentials" not found
    failed to get credentials secret
    github.com/ionos-cloud/cluster-api-provider-proxmox/pkg/scope.(*ClusterScope).setupProxmoxClient
        /workspace/pkg/scope/cluster.go:142
    github.com/ionos-cloud/cluster-api-provider-proxmox/pkg/scope.NewClusterScope
        /workspace/pkg/scope/cluster.go:119
    github.com/ionos-cloud/cluster-api-provider-proxmox/internal/controller.(*ProxmoxClusterReconciler).Reconcile
        /workspace/internal/controller/proxmoxcluster_controller.go:107
    sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Reconcile
        /go/pkg/mod/sigs.k8s.io/controller-runtime@v0.17.3/pkg/internal/controller/controller.go:119
    sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler
        /go/pkg/mod/sigs.k8s.io/controller-runtime@v0.17.3/pkg/internal/controller/controller.go:316
    sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem
        /go/pkg/mod/sigs.k8s.io/controller-runtime@v0.17.3/pkg/internal/controller/controller.go:266
    sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2
        /go/pkg/mod/sigs.k8s.io/controller-runtime@v0.17.3/pkg/internal/controller/controller.go:227
    runtime.goexit
        /usr/local/go/src/runtime/asm_amd64.s:1650
    Unable to initialize ProxmoxClient
    github.com/ionos-cloud/cluster-api-provider-proxmox/pkg/scope.NewClusterScope
        /workspace/pkg/scope/cluster.go:121
    github.com/ionos-cloud/cluster-api-provider-proxmox/internal/controller.(*ProxmoxClusterReconciler).Reconcile
        /workspace/internal/controller/proxmoxcluster_controller.go:107
    sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Reconcile
        /go/pkg/mod/sigs.k8s.io/controller-runtime@v0.17.3/pkg/internal/controller/controller.go:119
    sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler
        /go/pkg/mod/sigs.k8s.io/controller-runtime@v0.17.3/pkg/internal/controller/controller.go:316
    sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem
        /go/pkg/mod/sigs.k8s.io/controller-runtime@v0.17.3/pkg/internal/controller/controller.go:266
    sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2
        /go/pkg/mod/sigs.k8s.io/controller-runtime@v0.17.3/pkg/internal/controller/controller.go:227
    runtime.goexit
        /usr/local/go/src/runtime/asm_amd64.s:1650
 > controller="proxmoxcluster" controllerGroup="infrastructure.cluster.x-k8s.io" controllerKind="ProxmoxCluster" ProxmoxCluster="default/mk1-busi-cl" namespace="default" name="mk1-busi-cl" reconcileID="19414580-a689-45ae-8798-42eb43157644"
[14:07:42] k8sadmin@image-k get secret mk1-busi-cl-proxmox-credentials
NAME                              TYPE     DATA   AGE
mk1-busi-cl-proxmox-credentials   Opaque   3      31m

What did you expect to happen: I would have expected to upgrade capmox and then have to apply the required secret and add the required value into ProxmoxCluster.spec.credentialsRef.name I then would expect the capmox controller to communicate with proxmox.

Anything else you would like to add: [Miscellaneous information that will assist in solving the issue.]

Environment:

mcbenjemaa commented 2 weeks ago

@mkamsikad2 thanks for reporting this, While I tested this before I will go and give it another try to check if upgrading broke the ProxmoxClient.

mkamsikad2 commented 6 days ago

@mcbenjemaa I have upgraded to Capmox 0.5.1 and have now configured ProxmoxCluster.spec.credentialsRef.namespace

I can now build clusters and perform alpha rollouts. One thing that is still an issue is that all upgraded clusters are in a Failed state.

cluster.cluster.x-k8s.io/capi-management-v2                  Failed        66d     
cluster.cluster.x-k8s.io/dev1-uk-busi-cl                     Failed        66d     
cluster.cluster.x-k8s.io/mk1-busi-cl                         Provisioned   4m32s   
cluster.cluster.x-k8s.io/test1-uk-busi-cl                    Failed        60d     

NAME                                                                CLUSTER              READY   ENDPOINT
proxmoxcluster.infrastructure.cluster.x-k8s.io/capi-management-v2   capi-management-v2   true    {"host":"10.20.1.220","port":6443}
proxmoxcluster.infrastructure.cluster.x-k8s.io/dev1-uk-busi-cl      dev1-uk-busi-cl      true    {"host":"10.10.1.10","port":6443}
proxmoxcluster.infrastructure.cluster.x-k8s.io/mk1-busi-cl          mk1-busi-cl          true    {"host":"10.10.1.150","port":6443}
proxmoxcluster.infrastructure.cluster.x-k8s.io/test1-uk-busi-cl     test1-uk-busi-cl     true    {"host":"10.10.1.30","port":6443}

I'm not sure how to bring the cluster out of a failed state.