nsenter permission denied errors on 1.15.0 with signal reboot set

[mrocheleau]

After updating from 1.10.x to 1.15.0 and switching the manifest over to the kured-ds-signal.yaml one, using the signal reboot method and with privileged set to false on the container, kured comes up fine however is throwing persistent warnings of:

`time="2024-02-02T18:12:11Z" level=warning msg="nsenter: can't open '/proc/1/ns/mnt': Permission denied" cmd=/usr/bin/nsenter std=err`
`time="2024-02-02T18:12:15Z" level=info msg="Lock TTL set, lock will expire after: 30m0s"`
`time="2024-02-02T18:12:15Z" level=info msg="Lock release delay not set, lock will be released immediately after rebooting"`
`time="2024-02-02T18:12:15Z" level=info msg="PreferNoSchedule taint: "`
`time="2024-02-02T18:12:15Z" level=info msg="Blocking Pod Selectors: []"`
`time="2024-02-02T18:12:15Z" level=info msg="Reboot schedule: ---MonTueWedThu------ between 19:00 and 23:59 America/Los_Angeles"`
`time="2024-02-02T18:12:15Z" level=info msg="Reboot check command: [sh -c ! needs-restarting --reboothint] every 1h0m0s"`
`time="2024-02-02T18:12:15Z" level=info msg="Concurrency: 1"`
`time="2024-02-02T18:12:15Z" level=info msg="Reboot method: signal"`
`time="2024-02-02T18:12:15Z" level=info msg="Reboot signal: 39"`
`time="2024-02-02T18:12:15Z" level=warning msg="nsenter: can't open '/proc/1/ns/mnt': Permission denied" cmd=/usr/bin/nsenter std=err`
`time="2024-02-02T18:13:11Z" level=warning msg="nsenter: can't open '/proc/1/ns/mnt': Permission denied" cmd=/usr/bin/nsenter std=err`
`time="2024-02-02T18:13:12Z" level=warning msg="nsenter: can't open '/proc/1/ns/mnt': Permission denied" cmd=/usr/bin/nsenter std=err`
`time="2024-02-02T18:13:14Z" level=warning msg="nsenter: can't open '/proc/1/ns/mnt': Permission denied" cmd=/usr/bin/nsenter std=err`
`time="2024-02-02T18:13:15Z" level=warning msg="nsenter: can't open '/proc/1/ns/mnt': Permission denied" cmd=/usr/bin/nsenter std=err`
`time="2024-02-02T18:13:15Z" level=warning msg="nsenter: can't open '/proc/1/ns/mnt': Permission denied" cmd=/usr/bin/nsenter std=err`

These just keep repeating, one per pod/node.

My relevant portion of the daemonset manifest is:

  volumes:
    - name: sentinel
      hostPath:
        path: /var/run
        type: Directory
  containers:
    - name: kured
      image: ghcr.io/kubereboot/kured:1.15.0
      imagePullPolicy: IfNotPresent
      securityContext:
        privileged: false # Give permission to nsenter /proc/1/ns/mnt
        readOnlyRootFilesystem: true
        allowPrivilegeEscalation: false
        capabilities:
          drop: ["*"]
          add: ["CAP_KILL"]
      ports:
        - containerPort: 8080
          name: metrics
      env:
        # Pass in the name of the node on which this pod is scheduled
        # for use with drain/uncordon operations and lock acquisition
        - name: KURED_NODE_ID
          valueFrom:
            fieldRef:
              fieldPath: spec.nodeName
      volumeMounts:
        - mountPath: /sentinel
          name: sentinel
          readOnly: true
      command:
        - /usr/bin/kured
        - --reboot-sentinel=/sentinel/reboot-required
        - --reboot-method=signal
        ...
        - --reboot-sentinel-command=sh -c "! needs-restarting --reboothint"

Are these ignorable errors?

[mrocheleau]

It works fine if I switch the non-signal manifest with the rest of our custom command switches the same outside of the reboot-method=command in the non-signal one.

`time="2024-02-02T18:37:27Z" level=info msg="Reboot is probably not necessary." cmd=/usr/bin/nsenter std=out`
`time="2024-02-02T18:38:00Z" level=info msg="No core libraries or services have been updated." cmd=/usr/bin/nsenter std=out`

[ckotzbauer]

Hi @mrocheleau, you only can set privileged: false when you are using the signal-mode AND no custom reboot- or sentinel-command. Any command which should be executed, needs the nsenter command and privileged permissions. With privileged permissions you can of course use the new signal-mode, but it does not improve the security-configuration.

We should print a warning, when the pod is not privileged and a custom-command is configured.

[mrocheleau]

Hi @mrocheleau, you only can set privileged: false when you are using the signal-mode AND no custom reboot- or sentinel-command. Any command which should be executed, needs the nsenter command and privileged permissions. With privileged permissions you can of course use the new signal-mode, but it does not improve the security-configuration.

We should print a warning, when the pod is not privileged and a custom-command is configured.

Gotcha, ok this works then and if we ever switch distributions and no longer require a custom command we'll use the signal mode - thanks for the response!