search

rancher / elemental-operator

The Elemental operator is responsible for managing the OS versions and maintaining a machine inventory to assist with edge or baremetal installations.
Apache License 2.0
41 stars 17 forks source link

"registration:" components get dropped #108

Closed kkaempf closed 2 years ago

kkaempf commented 2 years ago

Running kubectl apply -f with this registration.yaml

apiVersion: elemental.cattle.io/v1beta1
kind: MachineRegistration
metadata:
  name: rpi-cluster-nodes
  namespace: fleet-default
spec:
  config:
    cloud-config:
      users:
      - name: root
        passwd: root
    elemental:
      registration:
        emulated-tpm: true
        emulate-tpm-seed: 4
        no-smbios: false
      install:
        automatic: true
        reboot: true
        debug: true
        device: /dev/sdb
  machineName: m-${System Information/Manufacturer}-${System Information/Product Name}-${System Information/UUID}

lets elemental-operator drop the registration: components. The Rancher manager UI only shows (note the empty registration value under spec.config.elemental)

apiVersion: elemental.cattle.io/v1beta1
kind: MachineRegistration
metadata:
  annotations:
    kubectl.kubernetes.io/last-applied-configuration: |
      {"apiVersion":"elemental.cattle.io/v1beta1","kind":"MachineRegistration","metadata":{"annotations":{},"name":"rpi-cluster-nodes","namespace":"fleet-default"},"spec":{"config":{"cloud-config":{"users":[{"name":"root","passwd":"root"}]},"elemental":{"install":{"automatic":true,"debug":true,"device":"/dev/sdb","reboot":true},"registration":{"emulate-tpm-seed":4,"emulated-tpm":true,"no-smbios":false}}},"machineName":"m-${System Information/Manufacturer}-${System Information/Product Name}-${System Information/UUID}"}}
  creationTimestamp: "2022-08-03T09:02:28Z"
  finalizers:
  - wrangler.cattle.io/machine-registration
  generation: 2
  managedFields:
  - apiVersion: elemental.cattle.io/v1beta1
    fieldsType: FieldsV1
    fieldsV1:
      f:metadata:
        f:finalizers:
          .: {}
          v:"wrangler.cattle.io/machine-registration": {}
      f:spec:
        f:config:
          f:elemental:
            f:system-agent: {}
    manager: elemental-operator
    operation: Update
    time: "2022-08-03T09:02:28Z"
  - apiVersion: elemental.cattle.io/v1beta1
    fieldsType: FieldsV1
    fieldsV1:
      f:status:
        .: {}
        f:conditions: {}
        f:registrationToken: {}
        f:registrationURL: {}
        f:serviceAccountRef:
          .: {}
          f:kind: {}
          f:name: {}
          f:namespace: {}
    manager: elemental-operator
    operation: Update
    subresource: status
    time: "2022-08-03T09:02:28Z"
  - apiVersion: elemental.cattle.io/v1beta1
    fieldsType: FieldsV1
    fieldsV1:
      f:metadata:
        f:annotations:
          .: {}
          f:kubectl.kubernetes.io/last-applied-configuration: {}
      f:spec:
        .: {}
        f:config:
          .: {}
          f:cloud-config:
            .: {}
            f:users: {}
          f:elemental:
            .: {}
            f:install:
              .: {}
              f:debug: {}
              f:device: {}
              f:reboot: {}
            f:registration: {}
        f:machineName: {}
    manager: kubectl-client-side-apply
    operation: Update
    time: "2022-08-03T09:02:28Z"
  name: rpi-cluster-nodes
  namespace: fleet-default
  resourceVersion: "15314"
  uid: 97801245-063d-4216-87dd-7d682199a7df
spec:
  config:
    cloud-config:
      users:
      - name: root
        passwd: root
    elemental:
      install:
        debug: true
        device: /dev/sdb
        reboot: true
      registration: {}
      system-agent: {}
  machineName: m-${System Information/Manufacturer}-${System Information/Product Name}-${System
    Information/UUID}
status:
  conditions:
  - lastUpdateTime: "2022-08-03T09:02:28Z"
    status: "True"
    type: Ready
  registrationToken: 4zgtpc66qkhhgvktsk75gzrqt4wrbpn2bptm4dvhf7s2xv8xh5rmk9
  registrationURL: https://192.168.0.33/elemental/registration/4zgtpc66qkhhgvktsk75gzrqt4wrbpn2bptm4dvhf7s2xv8xh5rmk9
  serviceAccountRef:
    kind: ServiceAccount
    name: rpi-cluster-nodes
    namespace: fleet-default
kkaempf commented 2 years ago

I can edit the machineRegistration in the Rancher Manager UI. Adding emulate-tpm: true gets accepted and stored. Adding emulated-tpm-hash: 4 does not get stored.

kkaempf commented 2 years ago

elemental-operator log doesn't show anything suspicious:

time="2022-08-03T12:06:15Z" level=info msg="Operator version 0.4.2, commit f4ba471, commit date git20220803"                                                                               │
│ time="2022-08-03T12:06:15Z" level=info msg="Starting controller at namespace cattle-elemental-system. Upgrade sync interval at: 1h0m0s"                                                    │
│ time="2022-08-03T12:06:15Z" level=info msg="Applying CRD managedosimages.elemental.cattle.io"                                                                                              │
│ time="2022-08-03T12:06:15Z" level=info msg="Applying CRD machineinventories.elemental.cattle.io"                                                                                           │
│ time="2022-08-03T12:06:15Z" level=info msg="Applying CRD machineregistrations.elemental.cattle.io"                                                                                         │
│ time="2022-08-03T12:06:15Z" level=info msg="Applying CRD managedosversions.elemental.cattle.io"                                                                                            │
│ time="2022-08-03T12:06:15Z" level=info msg="Applying CRD managedosversionchannels.elemental.cattle.io"                                                                                     │
│ time="2022-08-03T12:06:15Z" level=info msg="Applying CRD machineinventoryselectors.elemental.cattle.io"                                                                                    │
│ time="2022-08-03T12:06:15Z" level=info msg="Applying CRD machineinventoryselectortemplates.elemental.cattle.io"                                                                            │
│ time="2022-08-03T12:06:15Z" level=info msg="Creating event broadcaster for mos-bundle"                                                                                                     │
│ time="2022-08-03T12:06:15Z" level=info msg="Creating event broadcaster for machine-registration"                                                                                           │
│ time="2022-08-03T12:06:15Z" level=info msg="Starting fleet.cattle.io/v1alpha1, Kind=Bundle controller"                                                                                     │
│ time="2022-08-03T12:06:15Z" level=info msg="Starting elemental.cattle.io/v1beta1, Kind=ManagedOSImage controller"                                                                          │
│ time="2022-08-03T12:06:15Z" level=info msg="Starting elemental.cattle.io/v1beta1, Kind=MachineInventory controller"                                                                        │
│ time="2022-08-03T12:06:15Z" level=info msg="Starting elemental.cattle.io/v1beta1, Kind=MachineInventorySelector controller"                                                                │
│ time="2022-08-03T12:06:15Z" level=info msg="Starting elemental.cattle.io/v1beta1, Kind=MachineRegistration controller"                                                                     │
│ time="2022-08-03T12:06:15Z" level=info msg="Starting /v1, Kind=ServiceAccount controller"                                                                                                  │
│ time="2022-08-03T12:06:15Z" level=info msg="Starting management.cattle.io/v3, Kind=Setting controller"                                                                                     │
│ time="2022-08-03T12:06:15Z" level=info msg="Starting cluster.x-k8s.io/v1beta1, Kind=Machine controller"                                                                                    │
│ time="2022-08-03T12:06:15Z" level=info msg="Starting elemental.cattle.io/v1beta1, Kind=ManagedOSVersion controller"                                                                        │
│ time="2022-08-03T12:06:15Z" level=info msg="Starting /v1, Kind=Secret controller"                                                                                                          │
│ time="2022-08-03T12:06:15Z" level=info msg="Starting elemental.cattle.io/v1beta1, Kind=ManagedOSVersionChannel controller"                                                                 │
│ time="2022-08-03T12:06:15Z" level=info msg="Starting steve aggregation client"                                                                                                             │
Itxaka commented 2 years ago

I think this may have something to do with the fact that we dont have the config as part of the apis, so it doesnt generate some methods for it for complex classes. Something like missing deep copy or something like that.

Weirdly enough adding logging to the deepcopy shows that at that part the registration is already empty??? Not sure what is going on as this is from a yaml, there is nothing else in the middle.... need to keep digging

Itxaka commented 2 years ago

weirdly enough, if you apply the same yaml twice then registration gets the no-smbios key only?????

Itxaka commented 2 years ago

This is what is received on kubectl appy

config.Config{
  Elemental: config.Elemental{
    Install: config.Install{
      Firmware: \"\",
      Device: \"/dev/vda\",
      NoFormat: true,
      ConfigURLs: nil,
      ISO: \"\",
      SystemURI: \"\",
      Debug: true,
      TTY: \"\",
      PowerOff: false,
      Reboot: true,
      EjectCD: false,
    },
    Registration: config.Registration{
      URL: \"\",
      CACert: \"\",
      EmulateTPM: false,
      EmulatedTPMSeed: 0,
      NoSMBIOS: false,
      Labels: map[string]string(nil),
    },
    SystemAgent: config.SystemAgent{
      URL: \"\",
      Token: \"\",
      SecretName: \"\",
      SecretNamespace: \"\",
    },
  },
  CloudConfig: map[string]interface {}{
    \"users\": []interface {}{
      map[string]interface {}{
        \"name\": \"root\",
        \"passwd\": \"root\",
      },
    },
  },
}

with this test yaml:

apiVersion: elemental.cattle.io/v1beta1
kind: MachineRegistration
metadata:
  name: test-nodes
  namespace: fleet-default
spec:
  config:
    cloud-config:
      users:
      - name: root
        passwd: root
    elemental:
      registration:
        emulated-tpm: true
        emulate-tpm-seed: 4
        no-smbios: false
      install:
        automatic: true
        reboot: true
        debug: true
        no-format: true
        device: /dev/vda
  machineName: m-${System Information/Manufacturer}-${System Information/Product Name}-${System Information/UUID}
  machineInventoryLabels:
    cluster-id: kvm-cluster

So its getting the cloud-config and install parts correctly but not the registration? WTF is going on??

Itxaka commented 2 years ago

emulated-tpm IS emulate-tpm emulate-tpm-seed IS emulated-tpm-seed

Itxaka commented 2 years ago

using the proper fields makes it show, this is not a bug but a lack of docs or maybe a non-clear-enough-keys