No default OS affinity but can't run on Windows nodes

[rcmosher]

We're using the spicedb-operator to deploy to our K8S cluster. Our cluster allows for both Linux and Windows nodes. Occasionally we'll end up with a pod trying to start on a Windows node and failing. We can get around this with a patch, but since Windows nodes are not supported it would make sense to exclude it by default.

The patch in our SpiceDBCluster definition is as follows:

  patches:
  - kind: '*'
    patch:
      metadata:
        labels:
          added: via-patch
      spec:
        affinity:
          nodeAffinity:
            requiredDuringSchedulingIgnoredDuringExecution:
              nodeSelectorTerms:
              - matchExpressions:
                - key: kubernetes.io/os
                  operator: In
                  values:
                  - linux

[rcmosher]

I should add we've seen this patch ignored, probably due to deleting and recreating the SpiceDbCluster. And we've seen the spicedb-operator try to use a Windows node, which I don't think this patch controls. Nor does it seem to affect the spicedb-migration pod

[jzelinskie]

SpiceDB releases cut Windows binaries -- we should explore how much additional effort it'd just push Windows images.