[Feature] Ability to set priority (order) of extraManifests/custom CNI

[chrxmvtik]

Feature Request

Description

It would be great if we could specify a priority of files deployed using extraManifests, ie.

extraManifests:
  - url: https://example.com/manifest1.yaml
    priority: 100
  - url: https://example.com/manifest2.yaml
    priority: 101

Example use case:

We want to use Cilium as CNI with Gateway API support, which requires Gateway API manifests deployed before Cilium.

First thought would be to set extra manifests in ordered manner, ie.

extraManifests:
  - gateway-api.yaml
  - cilium.yaml

However when we check manifests order using talosctl get manifests command we can see something unusual:

talosctl -n 10.0.0.1 get manifests

NODE   NAMESPACE  TYPE       ID
10.0.0.1 controlplane   Manifest 99-cilium.yaml
10.0.0.1 controlplane   Manifest 99-gateway.yaml

Manifests got re-ordered by name and still, cilium got deployed firstly - so we fail to achive our goal.

There is a workaround to this issue, by hosting manifests on your own and applying numeric order to file names, ie.

00-gateway-api.yaml
01-cilium.yaml

But if you are willing to use manifests hosted on GitHub and maintanted by someone else, then there is no possibility to order manifests deployments.

[smira]

Unfortunately, this is not possible, Talos re-orders and splits individual resource definitions by prioritizing e.g. CRDs and Namespaces. But this seems to be something strange that order of applying manifests changes the final result, that seems not the way Kubernetes should work.