Passing CDI devices as annotations should be removed

[bart0sh]

Kubelet should stop using annotations to pass CDI device IDs to CRI runtimes. CDIDevices CRI field should be used for this purpose.

This should be done when Kubelet that doesn't support CRI field reaches EOL and two major CRI runtimes support CRI field.

Here is a PR comment that gives more details about this: https://github.com/kubernetes/kubernetes/pull/115891/files#r1112169121

• Kubelet started using CDIDevices CRI field in v1.28.0. Previous version 1.27 will reach EOL 2024-06-28.

• CRI-O started using it in v1.28.0. Previous version 1.27 will reach EOL 2024-06-28 together with Kubernetes 1.27.

• Containerd started using it in 1.7.2. Version 1.7 will reach EOL in a 6 months after Containerd 2.0 release https://github.com/containerd/containerd/blob/main/RELEASES.md. Previous version 1.6 doesn't support CDI injections and should not be considered.

So, safer option would be to wait until Containerd 1.7 reaches EOL. However it's relatively safe to do it earlier when 1.7.0 and 1.7.1 have little chances to be in real use.

Containerd v1.7.0 and v1.7.1 is not used by the Linux distributions from this long list, so it's safe to remove annotations support from the Kubelet in 1.31 in my opinion.

[bart0sh]

/sig node

[ffromani]

/triage accepted /priority important-longterm

[bart0sh]

/assign