search

ionos-cloud / cluster-api-provider-proxmox

Cluster API Provider for Proxmox VE (CAPMOX)
Apache License 2.0
181 stars 24 forks source link

cloud-init iso inject failed: unable to inject CloudInit ISO: unexpected end of JSON input #252

Closed justinas-b closed 2 months ago

justinas-b commented 2 months ago

What steps did you take and what happened: I am trying to provision my first cluster, however after first virtual machine is created i see below error:

E0701 21:46:45.868816       1 proxmoxmachine_controller.go:209] "error reconciling VM" err="cloud-init iso inject failed: unable to inject CloudInit ISO: unexpected end of JSON input" controller="proxmoxmachine" controllerGroup="infrastructure.cluster.x-k8s.io" controllerKind="ProxmoxMachine" ProxmoxMachine="default/mgmt-control-plane-wcsjg" namespace="default" name="mgmt-control-plane-wcsjg" reconcileID="69a7bb6b-77c2-4973-ba9a-176db57eb584" machine="default/mgmt-control-plane-wcsjg" cluster="default/mgmt"
E0701 21:46:45.871103       1 controller.go:329] "Reconciler error" err="failed to reconcile VM: cloud-init iso inject failed: unable to inject CloudInit ISO: unexpected end of JSON input" controller="proxmoxmachine" controllerGroup="infrastructure.cluster.x-k8s.io" controllerKind="ProxmoxMachine" ProxmoxMachine="default/mgmt-control-plane-wcsjg" namespace="default" name="mgmt-control-plane-wcsjg" reconcileID="69a7bb6b-77c2-4973-ba9a-176db57eb584"

I am using below configuration to bootstrap cluster:

export PROXMOX_URL="https://pve.mydomain.net"
export PROXMOX_TOKEN='svc-pve@MYDOMAIN.NET!aaaaaaaa-bbbb-cccc-dddd-ffffffffffff'
export PROXMOX_SECRET='11111111-2222-3333-4444-555555555555'
export PROXMOX_SOURCENODE="pve-01" 
export TEMPLATE_VMID="7000"
export ALLOWED_NODES="[pve-01]"
export VM_SSH_KEYS="ssh-rsa ...."
export CONTROL_PLANE_ENDPOINT_IP="10.11.1.100"
export NODE_IP_RANGES="[10.11.1.101-10.11.1.150]"
export GATEWAY="10.11.1.1"
export IP_PREFIX="24"
export DNS_SERVERS="[10.1.1.10,10.1.1.11]"
export BRIDGE="vmbr1100"
export BOOT_VOLUME_DEVICE="scsi0"
export BOOT_VOLUME_SIZE="50"
export NUM_SOCKETS="1"
export NUM_CORES="1"
export MEMORY_MIB="1024"
export EXP_CLUSTER_RESOURCE_SET="true"
export CLUSTER_TOPOLOGY="true"

clusterctl init --infrastructure proxmox --ipam in-cluster --core cluster-api:v1.7.2

clusterctl generate cluster mgmt \
    --infrastructure proxmox \
    --kubernetes-version v1.30.2 \
    --control-plane-machine-count 1 \
    --worker-machine-count 1 > ionos.yaml

What did you expect to happen: New virtual machines/cluster to be provisioned

Environment:

justinas-b commented 2 months ago

If i check the secrets, output of it looks like yaml, definitely not json:

❯ kubectl get secret "mgmt-control-plane-m6hs6" -o json -n default | jq -r '.data.value' | base64 -d
## template: jinja
#cloud-config

write_files:
-   path: /etc/kubernetes/pki/ca.crt
    owner: root:root
    permissions: '0640'
    content: |
      -----BEGIN CERTIFICATE-----
      MIIC6jCCAdKgAwIBAgIBADANBgkqhkiG9w0BAQsFADAVMRMwEQYDVQQDEwprdWJl
      cm5ldGVzMB4XDTI0MDcwMTIyMDUwN1oXDTM0MDYyOTIyMTAwN1owFTETMBEGA1UE
      AxMKa3ViZXJuZXRlczCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBALOp
      usD33Pao89iUpa0L+wYx+K38w4AkWiuRL32VSMwFDUMWFF37/Zm/DdNRyo4PVY1o
      o7UW16y750N2uWCEpl+OeFG0gix12Ok+Le5nM05XMhKcz9UZRlToiSM6+3KJKayq
      HA0UM49k8dMDYUZOf6CPgck3Am8XJQK17g6QAeJy/HwJrIw+u6JYPXMDxSrLK+mp
      IH0QdzuMy3kDquJq4vpyYEB5gnc74xx9UlynJGzeJzAnGNCRPJkOiMVHPAdsgLlQ
      D26q22mRJ6XVA3xu6Dpom0ToDdcgtCGCJrEiQgC8s8lcu74uJB0eu3U3+Av3vH6+
      HLncbUXPU6M8QhgP7H8CAwEAAaNFMEMwDgYDVR0PAQH/BAQDAgKkMBIGA1UdEwEB
      /wQIMAYBAf8CAQAwHQYDVR0OBBYEFEmbwj/IouwFFMQjrh6iA2klXp+6MA0GCSqG
      SIb3DQEBCwUAA4IBAQAslIB+uVkLlzSlAp4K+K68aoTl3mbiAHbRAn5eEnysxY7/
      JeYCzH05oD/NoQ5UNXcLt0uUyVDALcPqVBSRsixONYcVWT0XLz8QSx8rIY6e0ixU
      uM4bt8bFyhyIyFjTBKSI6UtPeOOhUvigB4QPrT9Xdp9BtGRObFWBw6/7+Mfvt45j
      wn5d+ww5lDYPCpwKTiePApkKRk2LoiYVHXtA4HvMvF7auqGMs+VkL9Q3lVYKUe5y
      ESJgxwcgrPgu4ezIDc+sSIUhqfx0fIxa06L2XJxZM5vliZRMwtM1f06KbKhACNDi
      BmF+yN13IaXhrDRyuLHHuMzyj+EgaKhEYvO3iL0I
      -----END CERTIFICATE-----

-   path: /etc/kubernetes/pki/ca.key
    owner: root:root
    permissions: '0600'
    content: |
      -----BEGIN RSA PRIVATE KEY-----
      MIIEogIBAAKCAQEAs6m6wPfc9qjz2JSlrQv7BjH4rfzDgCRaK5EvfZVIzAUNQxYU
      Xfv9mb8N01HKjg9VjWijtRbXrLvnQ3a5YISmX454UbSCLHXY6T4t7mczTlcyEpzP
      1RlGVOiJIzr7cokprKocDRQzj2Tx0wNhRk5/oI+ByTcCbxclArXuDpAB4nL8fAms
      jD67olg9cwPFKssr6akgfRB3O4zLeQOq4mri+nJgQHmCdzvjHH1SXKckbN4nMCcY
      0JE8mQ6IxUc8B2yAuVAPbqrbaZEnpdUDfG7oOmibROgN1yC0IYImsSJCALyzyVy7
      vi4kHR67dTf4C/e8fr4cudxtRc9TozxCGA/sfwIDAQABAoIBAAsdFrZVHtALR3pA
      R7dapF6PvMhWeGl21jUXuA5Wq7gjH9zkgsIFknxuOxa1WohLHomuzWGoVhMTVIzx
      zSnRStFiaML1HZBOgep7tC+pJp1T+373QEv4UkdyUvlg7qj5HyDVLQMKGt5l6ieZ
      b+z/chmmhurBgl8I2Xs/ccho1BUD2TmU2md1L/WNpGG2JyViKraciqph7SoD2CrI
      3e2eVPxwvRtF4ZkTkwxLGBVL7vhJxjXKBW+ybXkN26b9PdGGS97oBK2SHRFqjb5L
      Ae7cl/d/ZbkvGnNKwZUJBpw+0/zxiP3Hfmso7LYroyHOgHJcqVfCSIfBIZbVeX1+
      o11wd5ECgYEA7eCeHjZZd3jYiUP1UZQyH3tUcVava+IUYVTzn+dTjxFmi4Xrlvxh
      vNpqRicfUL6nE9t5XZyLeMJlVQbuWCS/nDEpyvJOj5HWP3OfRbo29Lu8y6krfwID
      LxEv1Kygj1fDW59sGTzHGJhNqRU9KCj6PkkugPsvxAG1BH7RluewzP0CgYEAwVm9
      5rjvyRqfB3P60la8Z7FwxG93bVuuFBggKeR3x2vnvTyzwNulegADXQNj8Tg1Zvc0
      qcrTzfmPhQYnFDVD2nqEQdwKy67Vm7vzS7j3PFx8Pzdn9p88ySZigi6ZLIyWdaJE
      vBoieZQz4OcDIavbCHnIrOsm1HYMZW9thOqq1isCgYBj6BPgsoju4aVoOU+ePLS0
      fjLgCz1P/Tr8ChbsUNbKXs+Hx+5DkqpmDAHYhvPjEL/RWjtl8qqtRFAU3v1Zrc2A
      BzH7V7thFyo5qhD50xTjwrmve9SV2j2GFhzVoM8qhPy6zYE0NE+WZ9kuK4ptBJDd
      ehT6/QR5uRHS2oSicZrS7QKBgCR4kxOUSta6SMkVHXjTl5JPrB/Bjtt38awkqNx+
      FgHt3raZXqbAR8RtgDTpUM7NdXVIqAxnD/DIZEQJVxG+wl/RwGTT5s1o2y4XE9ty
      tmASafVatCeW/dZLgUVhGM+IbVEfHKwGmf4o8Gg56yNM0XIV1r6XVKm64re3fwcV
      ZMglAoGAaluHTmSKg/Qr4Tiohw9H4vT9PumGcb7A4o9oPeMQ6KtZG78egIVTXoQP
      I1srmVwxwhQnfAVSkKhwomcuUrn/t2e7KZlYjLCAWWn0FGZJSGVgRaMK12vruRCY
      rmYUM4OB81mS4OHVs3ZxD6g2TUO2WBZNCetg4FggJE2A7olOq70=
      -----END RSA PRIVATE KEY-----

-   path: /etc/kubernetes/pki/etcd/ca.crt
    owner: root:root
    permissions: '0640'
    content: |
      -----BEGIN CERTIFICATE-----
      MIIC6jCCAdKgAwIBAgIBADANBgkqhkiG9w0BAQsFADAVMRMwEQYDVQQDEwprdWJl
      cm5ldGVzMB4XDTI0MDcwMTIyMDUwN1oXDTM0MDYyOTIyMTAwN1owFTETMBEGA1UE
      AxMKa3ViZXJuZXRlczCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBAMEX
      2zbPJ3iQpsOSN1leDcebdUnMAVRwC6YSEBUhdlt0BEx5Y2AvdIG2dmw2lb16DCe8
      7ELX4z4ZiGMM6JiCekIvXcCPj9oTHpfd0SZeUeGP1+oLdyZe1NRKPCpEmUj52Ox+
      DmtVJqXuxI2D4dLK8KvZZhwhw/UyEBe6gYQxTjqPew0dL6McKbGVLNoYlhCqYo/H
      JhLjDtKZV3/oNRXzvCriiWHYeq+ExaGnVrvkivbKKHVnSTQPxdDc2SIAVxQ+N849
      +1YhQrhRemk4ZZmV0DnkQdVb7VcaNRS8rpyXwd5piw0o3VYgiyGO0QgBdSlIYaKc
      f2sPSxsAlFuf2SwVr2UCAwEAAaNFMEMwDgYDVR0PAQH/BAQDAgKkMBIGA1UdEwEB
      /wQIMAYBAf8CAQAwHQYDVR0OBBYEFK/T9hOZScQmQPI5LaR1EETpOfQsMA0GCSqG
      SIb3DQEBCwUAA4IBAQBtO29FPTgvbGnFcBUtmkGR05291yqMJ8ciFsG8MFKCVOY8
      qT6r9B3OPc24ihsqL2eAn4ml3IsyQg3TAWUySPDjxLJUc90VGqWjyya6pE6cYYm8
      bdLaZwfWr64yEvt0eHF5+HgIMdnKEsGpuxOCmXunDiR8aExs07P+3yfyRwonhiKF
      vMtSzOsx26Vi0Da0SgEZPuCp0KEIefXlOinP3RJ3SyJZPQoF5dtML0pJxsEVblNI
      5eP0xqEo/aDGO2pA1qtMDv03tKigKAkr+xMUhFkAL9HzGXK7CV5BrxasZ/M71/yq
      UQ8ycoySabTTqtYW3toh9TMx70n3hYkNNWNUmmD/
      -----END CERTIFICATE-----

-   path: /etc/kubernetes/pki/etcd/ca.key
    owner: root:root
    permissions: '0600'
    content: |
      -----BEGIN RSA PRIVATE KEY-----
      MIIEpAIBAAKCAQEAwRfbNs8neJCmw5I3WV4Nx5t1ScwBVHALphIQFSF2W3QETHlj
      YC90gbZ2bDaVvXoMJ7zsQtfjPhmIYwzomIJ6Qi9dwI+P2hMel93RJl5R4Y/X6gt3
      Jl7U1Eo8KkSZSPnY7H4Oa1Umpe7EjYPh0srwq9lmHCHD9TIQF7qBhDFOOo97DR0v
      oxwpsZUs2hiWEKpij8cmEuMO0plXf+g1FfO8KuKJYdh6r4TFoadWu+SK9soodWdJ
      NA/F0NzZIgBXFD43zj37ViFCuFF6aThlmZXQOeRB1VvtVxo1FLyunJfB3mmLDSjd
      ViCLIY7RCAF1KUhhopx/aw9LGwCUW5/ZLBWvZQIDAQABAoIBAA8+iOn81Mrp8KCE
      DKabjE7z8QCQgJJf/itwRaDFcYiq0iQPeU8AsBwFgbftX0kEN61zdtuYG7KtZV0c
      ichwlKJhBu/K5bp7w/CNbJOw4poCNzOdEB0RY5AXBET6AXgHcZ7KHO1QXaq//PiH
      Lu7rYrDOiwcxizzyZN7atJrxIZHx2s5eYPEkWDnDvFvo0lyHOyQtSEpi0FQkBzgg
      EW/g2Y2MOObzBqBhzpc8bn8l4XO3JfiMu7+wFcvgUyNTUkF6omcf8NPTRSmqx6YK
      QuqZZtV6+2JADc/QpG0oee5sRjPJ0kP8UL+TFVyc0f7/O8Kt5U/jnhiIq7QKwQ/j
      cr3G+8kCgYEA5luRWCHOS8svZFK1dftYxMV3wz8wcpSQ0agqOZq9ra8iYtdFEtSC
      pXPmdc3+EITCKZwcZY4v7Y34WqdXCjvzleDB5dfm3pDBzB0Ir5s/o2ncjD45e+eg
      QsyTviAV7s88i8ikOfXjBxgAvGJ12Cqtc/E9KtAywP/QPX9XGSxDdDcCgYEA1pZf
      pHekSXXJhnGO8xcexOhF3PSYc1N9OVeHrU72p1nrLJ5e4YCKcHrKMZl4zhVg37FH
      go0LaTv1qYreEIp3wzBct5hZBNAvh3dV9a3FO2odPTLSFmbxkpnaacMm2ketvJSk
      3JQAtdPQDe06VSgVu9RbX3uNqNiq5Dy9yL7rY0MCgYEAkQlQLsxCJkiM0z6cD+es
      EWA/MohjiZDuyYJ264P5LxTYsqDVVbDFfUTG8p86+s2LrkUjP/HRTWGY4AMCc2gh
      ytkTXBKewr0jt88D/ATkgZASk55N9Sx2I/jIgOmUxaJnfZ9bfz7IrVu4guP/TJY6
      VYirOXySqB+ozj+R6ECNS+sCgYABAivo5fcLsVYFH7uAQPeQDVdmtm+Q+y93eYXV
      tq1KFA4a45Omk6gS50DUAtiif085PegMIQWQtQtuhIyOn1f4vt1wr3Kbpy7SACmT
      QgHPSaxhgT+jg1BXVAnGvXgNRpD2dexgbQGj2R79McQ4ajvP6iuTw0zjIkGLWQ27
      R8praQKBgQCkS91eoTCFaKIUyKJXptCp0nOAlgqsIi/hCQTki53e7jaVMjGQTJnY
      I5ZA88si40qSkWnU1q0YQb4F7LHZSBE7QudDJ8bzH4t5kMcxg2QaoatMNSSRANVH
      yirGacFXqLSrCpVMpmQpVfEzdoPK1qB5Dz1MFSh0sOKaOQaSPg34qg==
      -----END RSA PRIVATE KEY-----

-   path: /etc/kubernetes/pki/front-proxy-ca.crt
    owner: root:root
    permissions: '0640'
    content: |
      -----BEGIN CERTIFICATE-----
      MIIC6jCCAdKgAwIBAgIBADANBgkqhkiG9w0BAQsFADAVMRMwEQYDVQQDEwprdWJl
      cm5ldGVzMB4XDTI0MDcwMTIyMDUwN1oXDTM0MDYyOTIyMTAwN1owFTETMBEGA1UE
      AxMKa3ViZXJuZXRlczCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBAL/R
      XrgFmbvh0CZqSmOVpuhD0fJgii/z4ruDi8gtwsWZcWSTkQw3Gs7sSMAjQOsHbR7X
      Z7Sissb+4o7FonQH9LDK94+co9eiBI8OPf4Pv++AxJN4CAP73ywE1o8Hv623jDP7
      NY5auUcTEvUu50pa+/+sXiJQuLN8Waym8jcQ/GbFEL17OtBM36xaZx4ZQmlvY1P8
      hE5WEs2aiITiIZq9Bkr5EVwVP8RhP7gtveTEzrQOYLqWSXStGPx4/ZhIpc8u3Ad9
      V+H66i0hTfZdvSEYdj8DfDmF0znvB8/MRAdfWhBZQVeYNzo09FpNQYB96hXPigIj
      D4fiF1RQpgSUuzCxtokCAwEAAaNFMEMwDgYDVR0PAQH/BAQDAgKkMBIGA1UdEwEB
      /wQIMAYBAf8CAQAwHQYDVR0OBBYEFL6pVeKbgpyl0c0yLRMN2EPN4fq4MA0GCSqG
      SIb3DQEBCwUAA4IBAQBQp3LMKhLot+RarvDyTBCgRZxOQag+dMUD7OF/DTS/8vne
      wQA6+zdOt9OnlWdWmicAHthn9bzfjd3TKGh7SzC8S0mwQFokc28ayjEu1JLlrtTm
      6/k9KvGSR8mB9W+hAZ6gAaQPM1jEP9SPXogPhlMipzIbcXx/lxDcq2dGXTP44AZZ
      3Sw6gtqayS6JRa70hKNXokFa7ncfb77krTGsMpBXTRMHoafSfuKiSAj9KqvxI3mi
      63TrBZ7aDVnRJjaD98BUxolUCzdZ5Xf1RkH6Dtaf7EUax0qi0LDCWwGt2r+OeFyr
      8G+mHEwYSCNv6MplvfSNlt2pNTemB09FTkkDpvm9
      -----END CERTIFICATE-----

-   path: /etc/kubernetes/pki/front-proxy-ca.key
    owner: root:root
    permissions: '0600'
    content: |
      -----BEGIN RSA PRIVATE KEY-----
      MIIEpAIBAAKCAQEAv9FeuAWZu+HQJmpKY5Wm6EPR8mCKL/Piu4OLyC3CxZlxZJOR
      DDcazuxIwCNA6wdtHtdntKKyxv7ijsWidAf0sMr3j5yj16IEjw49/g+/74DEk3gI
      A/vfLATWjwe/rbeMM/s1jlq5RxMS9S7nSlr7/6xeIlC4s3xZrKbyNxD8ZsUQvXs6
      0EzfrFpnHhlCaW9jU/yETlYSzZqIhOIhmr0GSvkRXBU/xGE/uC295MTOtA5gupZJ
      dK0Y/Hj9mEilzy7cB31X4frqLSFN9l29IRh2PwN8OYXTOe8Hz8xEB19aEFlBV5g3
      OjT0Wk1BgH3qFc+KAiMPh+IXVFCmBJS7MLG2iQIDAQABAoIBAQC375EgbV5p57Ot
      4qbKr6qJdt9PyHs6WGfG3/GeVyM6UFWH2GlyiTTPz4xDkoqh1VEyDeSINquwXqHp
      vmJ1WVuuXDLgJ5iFIfu1IjcX9QpcI+JcdvFYwKHUmB8WsJuBfUyYW/+TLjQUj5nS
      OtPXXT/gf+0UgstacZecE8qYO1WNRSbAo1vrMlJi0FqtJXWhCUEG9QPuIsGdkO8s
      wPE8wMLJ9338laIgk7PgvRL9Zbr1GF30PF8mUABfyZyQwG9SvqlRCZBjmdhv9xVK
      +HrOsiJqSMwMHWS8s4WMPj19gOaMB/RVG0CA7SLrO8GaucDaqSIxDVapGI4f1n/a
      jFfGGqwBAoGBAPu+0khBRwVdgHIqO2EYD6VZQzKPyywaLqAMZ99CVaU2WggyoCL+
      zkPNANZJB7OjiYKTOtMsuEI7LTRamyyj6vayXAN+tOFhu1Bjy/npryz/Ie5zJiyt
      dJteFu/zijY4DTVADpdb/W0Xfm8u1Bxl98O4GTR9Wl0/w8P+RdoYoDEhAoGBAMMP
      RYRuaKexeEADsB1JffnYBmVduiOwbeSaG8r7iMz8mBg3s0Of7UTd05SUKK3p53wf
      6udFbyppM2y2URFk0zATZ0qnxAnDynjx2fBrlPNLmZKDSLiT+PjvShrR/mnPFKD4
      AoFlnbHBSi6ymc3dS8EIlg2XrLwEh8zi+zLQ+5BpAoGBAMyYArIySxPzL4aZB/Ua
      5uxMpS4catSG2sm8C1cIxC4nA/iRnE+UofG8VCfS5m16RTFXkoE5YO6CxXQ9Bsu6
      1CR2RA2bFkntcZwd3xJWdsWs+9icwLOirJfwWdfdJ8B0wVc7niAo5MeaRmj5t+z0
      ee4YWnOsRJpwThulNQPQ66/BAoGAXfpCnP5wz7JyZC0iscmRt0j8M+RHZSCAoa4a
      C97KVzCvuqNDbCZmM1LIuIH1Y20ZSTU7gLd0SrOqwgrI3h98nGIM1uwjSC0bpM+p
      F+tNNTbMQ8maxMsOHilzw54+/A14O+Ghm2Qw6M5ehkcoUQX4lIUlfmEAKQnpv63w
      JtiHC1kCgYBXLVlisIWGFDR0Sc0H8gpta5XXrvx5BOU2UDjNc0OLyVgl2uIy59mF
      RAZrIOASwc3b9N3RCLyW2jXEk2dFNUWLm8G0mohcdoJrd8QFNNDPYbQ/d4iyIUkh
      BHeS2urNTEfaMYkdkO9CGIcqw9U8oPK4X1jN/nIUXfXekSF6eK6QZg==
      -----END RSA PRIVATE KEY-----

-   path: /etc/kubernetes/pki/sa.pub
    owner: root:root
    permissions: '0640'
    content: |
      -----BEGIN PUBLIC KEY-----
      MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAyZs7Cihf0EPCyWlaBPmk
      N9DjyYcj+4radAMzTN7pvzzhDf6PQLvJlcdyWKMM+aSUQt/BlRvFBehft+nT/l6N
      OUoxhuCQWcALsHgqdkyXnq1zucBrZqAT0SJVURGK5Vn4QNnuzvv1M8fmnPvnQrUS
      TQB8PoqQj4fEAgyy4B6QBE7J5FpPp5Xcj5vLN9nO6sDBuJO1JK6m6aDDeXF/i8OU
      XRGzA2t+ZGarDlICt0ygX/ufJLdFdUf0U8P239CrGAwCPdd0b0nYhnEsYO9t5QzW
      i40fDERytF+FuG2PKE97hmpr+zeukGOPZT1IkL1KyFAY3ZRxNZKaNT+MqzvZrJhG
      gQIDAQAB
      -----END PUBLIC KEY-----

-   path: /etc/kubernetes/pki/sa.key
    owner: root:root
    permissions: '0600'
    content: |
      -----BEGIN RSA PRIVATE KEY-----
      MIIEowIBAAKCAQEAyZs7Cihf0EPCyWlaBPmkN9DjyYcj+4radAMzTN7pvzzhDf6P
      QLvJlcdyWKMM+aSUQt/BlRvFBehft+nT/l6NOUoxhuCQWcALsHgqdkyXnq1zucBr
      ZqAT0SJVURGK5Vn4QNnuzvv1M8fmnPvnQrUSTQB8PoqQj4fEAgyy4B6QBE7J5FpP
      p5Xcj5vLN9nO6sDBuJO1JK6m6aDDeXF/i8OUXRGzA2t+ZGarDlICt0ygX/ufJLdF
      dUf0U8P239CrGAwCPdd0b0nYhnEsYO9t5QzWi40fDERytF+FuG2PKE97hmpr+zeu
      kGOPZT1IkL1KyFAY3ZRxNZKaNT+MqzvZrJhGgQIDAQABAoIBACWbWEMHzv+pdC9g
      AJPHAZQL2kHgfJzShPd5FVu0hUPQFmcm4hAAN5VB9BmGqMf0pFdpJ9IpsPHuK9TB
      wHLFrnBOX7i6MRMc2NFdSQZyOSI3VswCkcY9/BRZ5DJQJGmuA7oNLRfdNhsnoeQM
      tVHjNvu4KHT+IgM0HyMGvWNDMDgtKXecHy2npV9QUYGuvbCy6r5oVFWXNwQqR0o2
      SEPrw2J0/6R+isHQoRUtHFFvLooQjxBu8/6yuHcu/66UOmBX3h4fly0oLLlatauU
      PpJ6gcM1N8cL9iiSl3Yjq7ZF8dB62l4YC3oayqIOovgBCkA2KaDTYyOm1MDenW5W
      ij2nrVUCgYEA2mqol8aTmo3SZt/QmuT3xT20rRmFc3rhS+ShPefRUEXJg1Ty591z
      9UZV0SS4uaisLVMAteKgKSyUHSMkcSviN2zQcmHWJzA9O5CPUpogs7dIlaBRsuj+
      MiW8NgzgW8HKEfvcZcUaqjnpmzV5V2Zn7EyuiTnFDzn8qOgCT0L9Y+sCgYEA7EwS
      svZp9KHHsxS02KYtmNZcDYLDP7eCusUMg4P9gTZUOZ43mo+sGabE5lz1hv2woK9D
      7CizlGqFwIiC6USXN/sFAyrt16BNjPZyiABp6/h0IqOikB9LdrI6B/aQkIZbD95Y
      bKmQie8NzCr0fxKTLd1UzxA4f95FivmbvYutYEMCgYBQjUmEQD4rn+Za/mxG0+71
      FV7PADyRbF4uyTEWTaqKelJpVykGjku3x6raqTpfbm+hf50hd2UuCjzNyioketTB
      MCZoKOD8LvRGRXg2NJWe8heY5zl6TCeDgxs4q+MKtBu8KjEMb3b4FMes8DnCHAhz
      w4wTG/UrOm2ONZK0fLZK5QKBgGANTEyS/d8DJDoXYKvwYPiPqjuTu+panjltf/i1
      3oZ3nyUMu22ImVyT/dXmU6fb18B5fhKxOM5LvrLUyuInj5Ch6Rewi53q+5shUs0V
      3fidtheejKYpEnPdXtl90KXu93Ur3KXPB4IKOoGTtG8zDcDHB4JkBZN0trAXZsdV
      z9QhAoGBAMLz3oMx1DUmVkiQSaBYWl/kmrFPKmnS3Uh+V7NA5yiq9OTntfasMLO6
      5V2sbS3T/PtDhfmaZ90YN4CE8yQixeo/Q0yr8YjJISWkGsFDLUbOBKgt1TzxZUZm
      GbdeItGrxrjjEzMl+d11zkqBZJllVu09b+3xr2Eh6ei60ICflPnK
      -----END RSA PRIVATE KEY-----

-   path: /etc/kubernetes/manifests/kube-vip.yaml
    owner: root:root
    content: |
      apiVersion: v1
      kind: Pod
      metadata:
        creationTimestamp: null
        name: kube-vip
        namespace: kube-system
      spec:
        containers:
        - args:
          - manager
          env:
          - name: cp_enable
            value: "true"
          - name: vip_interface
            value: ""
          - name: address
            value: 10.11.1.100
          - name: port
            value: "6443"
          - name: vip_arp
            value: "true"
          - name: vip_leaderelection
            value: "true"
          - name: vip_leaseduration
            value: "15"
          - name: vip_renewdeadline
            value: "10"
          - name: vip_retryperiod
            value: "2"
          image: ghcr.io/kube-vip/kube-vip:v0.7.1
          imagePullPolicy: IfNotPresent
          name: kube-vip
          resources: {}
          securityContext:
            capabilities:
              add:
              - NET_ADMIN
              - NET_RAW
          volumeMounts:
          - mountPath: /etc/kubernetes/admin.conf
            name: kubeconfig
        hostAliases:
        - hostnames:
          - localhost
          - kubernetes
          ip: 127.0.0.1
        hostNetwork: true
        volumes:
        - hostPath:
            path: /etc/kubernetes/admin.conf
            type: FileOrCreate
          name: kubeconfig
      status: {}

-   path: /etc/kube-vip-prepare.sh
    owner: root:root
    permissions: '0700'
    content: |
      #!/bin/bash

      # Copyright 2020 The Kubernetes Authors.
      #
      # Licensed under the Apache License, Version 2.0 (the "License");
      # you may not use this file except in compliance with the License.
      # You may obtain a copy of the License at
      #
      #     http://www.apache.org/licenses/LICENSE-2.0
      #
      # Unless required by applicable law or agreed to in writing, software
      # distributed under the License is distributed on an "AS IS" BASIS,
      # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
      # See the License for the specific language governing permissions and
      # limitations under the License.

      set -e

      # Configure the workaround required for kubeadm init with kube-vip:
      # xref: https://github.com/kube-vip/kube-vip/issues/684

      # Nothing to do for kubernetes < v1.29
      KUBEADM_MINOR="$(kubeadm version -o short | cut -d '.' -f 2)"
      if [[ "$KUBEADM_MINOR" -lt "29" ]]; then
        exit 0
      fi

      IS_KUBEADM_INIT="false"

      # cloud-init kubeadm init
      if [[ -f /run/kubeadm/kubeadm.yaml ]]; then
        IS_KUBEADM_INIT="true"
      fi

      # ignition kubeadm init
      if [[ -f /etc/kubeadm.sh ]] && grep -q -e "kubeadm init" /etc/kubeadm.sh; then
        IS_KUBEADM_INIT="true"
      fi

      if [[ "$IS_KUBEADM_INIT" == "true" ]]; then
        sed -i 's#path: /etc/kubernetes/admin.conf#path: /etc/kubernetes/super-admin.conf#' \
          /etc/kubernetes/manifests/kube-vip.yaml
      fi

-   path: /run/kubeadm/kubeadm.yaml
    owner: root:root
    permissions: '0640'
    content: |
      ---
      apiServer: {}
      apiVersion: kubeadm.k8s.io/v1beta3
      clusterName: mgmt
      controlPlaneEndpoint: 10.11.1.100:6443
      controllerManager: {}
      dns: {}
      etcd: {}
      kind: ClusterConfiguration
      kubernetesVersion: v1.30.2
      networking:
        podSubnet: 192.168.0.0/16
      scheduler: {}

      ---
      apiVersion: kubeadm.k8s.io/v1beta3
      kind: InitConfiguration
      localAPIEndpoint: {}
      nodeRegistration:
        imagePullPolicy: IfNotPresent
        kubeletExtraArgs:
          provider-id: proxmox://'{{ ds.meta_data.instance_id }}'
        taints: null

-   path: /run/cluster-api/placeholder
    owner: root:root
    permissions: '0640'
    content: "This placeholder file is used to create the /run/cluster-api sub directory in a way that is compatible with both Linux and Windows (mkdir -p /run/cluster-api does not work with Windows)"
runcmd:
  - "/etc/kube-vip-prepare.sh"
  - 'kubeadm init --config /run/kubeadm/kubeadm.yaml  && echo success > /run/cluster-api/bootstrap-success.complete'
users:
  - name: root

Though i am not sure from where this error comes

justinas-b commented 2 months ago

After enabling verbose mode on controller, i see below details:

capmox-controller-manager-f7688858-t7624 manager I0701 22:23:30.418890       1 logger.go:52] SEND: GET - https://pve.mydomain.net/api2/json/nodes/pve-01/storage
capmox-controller-manager-f7688858-t7624 manager I0701 22:23:30.558853       1 logger.go:52] RECV: 200 - 200 OK
capmox-controller-manager-f7688858-t7624 manager I0701 22:23:30.559128       1 logger.go:52] BODY: {"data":[{"enabled":1,"used":0,"total":0,"content":"images,rootdir","shared":1,"storage":"vmdata","type":"rbd","avail":0,"active":0},{"active":0,"avail":0,"storage":"local-lvm","type":"lvmthin","content":"rootdir,images","shared":0,"total":0,"used":0,"enabled":0},{"used_fraction":0.235735865291633,"active":1,"used":24283648000,"enabled":1,"storage":"local","type":"dir","avail":78728458240,"content":"iso,vztmpl,backup","total":103012106240,"shared":0}]}
capmox-controller-manager-f7688858-t7624 manager I0701 22:23:30.619874       1 logger.go:52] RECV: 400 - 400 Bad Request
capmox-controller-manager-f7688858-t7624 manager I0701 22:23:30.619947       1 logger.go:52] BODY:
capmox-controller-manager-f7688858-t7624 manager E0701 22:23:30.620397       1 proxmoxmachine_controller.go:209] "error reconciling VM" err="cloud-init iso inject failed: unable to inject CloudInit ISO: unexpected end of JSON input" controller="proxmoxmachine" controllerGroup="infrastructure.cluster.x-k8s.io" controllerKind="ProxmoxMachine" ProxmoxMachine="default/mgmt-control-plane-m6hs6" namespace="default" name="mgmt-control-plane-m6hs6" reconcileID="7929c839-b5c7-450a-8dd7-cdd3b3eb0a3c" machine="default/mgmt-control-plane-m6hs6" cluster="default/mgmt"
capmox-controller-manager-f7688858-t7624 manager E0701 22:23:30.623296       1 controller.go:329] "Reconciler error" err="failed to reconcile VM: cloud-init iso inject failed: unable to inject CloudInit ISO: unexpected end of JSON input" controller="proxmoxmachine" controllerGroup="infrastructure.cluster.x-k8s.io" controllerKind="ProxmoxMachine" ProxmoxMachine="default/mgmt-control-plane-m6hs6" namespace="default" name="mgmt-control-plane-m6hs6" reconcileID="7929c839-b5c7-450a-8dd7-cdd3b3eb0a3c"
justinas-b commented 2 months ago

Seems to be related to https://github.com/luthermonson/go-proxmox/issues/115, which looks like to be addressed in v.0.1.0 version of go-proxmox package. And it seems that this package version is used in v0.5.1 providers version (https://github.com/ionos-cloud/cluster-api-provider-proxmox/commit/f9c532bfe946bb05f33f20145ea01ccfb4e622b3)m but i am still experiencing this issue

justinas-b commented 2 months ago

Looking at the logs it seems that responce with storage devices contains 3 devices, and i am wondering if those a filtered and only single storage device which meets requirements for ISO images is passed, or are all three storage devices passed to request which causes this error

I0702 07:01:45.957598       1 logger.go:52] SEND: GET - [redacted secret capmox-manager-credentials:url]/api2/json/nodes/pve-01/storage
I0702 07:01:46.119779       1 logger.go:52] RECV: 200 - 200 OK
I0702 07:01:46.119816       1 logger.go:52] BODY: {"data":[{"active":1,"used_fraction":0.236132971549267,"shared":0,"storage":"local","enabled":1,"content":"backup,iso,vztmpl","total":103012106240,"used":24324554752,"avail":78687551488,"type":"dir"},{"enabled":1,"used":265559838121,"content":"rootdir,images","total":9202288922025,"shared":1,"storage":"vmdata","used_fraction":0.0288580200394928,"active":1,"type":"rbd","avail":8936729083904},{"type":"lvmthin","avail":0,"used":0,"content":"images,rootdir","total":0,"enabled":0,"storage":"local-lvm","shared":0,"active":0}]}
I0702 07:01:46.192229       1 logger.go:52] RECV: 400 - 400 Bad Request
I0702 07:01:46.192307       1 logger.go:52] BODY: 
E0702 07:01:46.192680       1 proxmoxmachine_controller.go:209] "error reconciling VM" err="cloud-init iso inject failed: unable to inject CloudInit ISO: unexpected end of JSON input" controller="proxmoxmachine" controllerGroup="infrastructure.cluster.x-k8s.io" controllerKind="ProxmoxMachine" ProxmoxMachine="default/mgmt-control-plane-gtbbq" namespace="default" name="mgmt-control-plane-gtbbq" reconcileID="4ae3bf02-d7fe-4bd5-9da1-aea443f33d96" machine="default/mgmt-control-plane-gtbbq" cluster="default/mgmt"
E0702 07:01:46.193386       1 controller.go:329] "Reconciler error" err="failed to reconcile VM: cloud-init iso inject failed: unable to inject CloudInit ISO: unexpected end of JSON input" controller="proxmoxmachine" controllerGroup="infrastructure.cluster.x-k8s.io" controllerKind="ProxmoxMachine" ProxmoxMachine="default/mgmt-control-plane-gtbbq" namespace="default" name="mgmt-control-plane-gtbbq" reconcileID="4ae3bf02-d7fe-4bd5-9da1-aea443f33d96"
justinas-b commented 2 months ago

Managed to connect with dlv debug - it seems error is caused in [task, err := storage.Upload("iso", iso.Name())]](https://github.com/luthermonson/go-proxmox/blob/215b6c1cd08a8743314218f35c05898740dcc3c2/virtual_machine.go#L148-L151). Exact exception is invalid character 'B' looking for beginning of value"

image

But i cannot understand from where this 'B' comes in the error

justinas-b commented 2 months ago

Hmm, i see that response to upload operation is actually "Bad Gateway", which probably explains B letter

image
justinas-b commented 2 months ago

OK, it seems issue was related to privileges on proxmox. User i am using didn't had necessary privileges to upload ISO image which resulted in "unexpected end of JSON input" error