Add default resource request/limits for SpiceDB Operator

[jawnsy]

Summary

Add CPU and memory request/limits for SpiceDB Operator deployment

Details

The SpiceDB Operator manifest is missing resource specs, which means that the resulting pod will have BestEffort quality of service.

This is typically a low-priority issue because the pod is mainly idle, is not very resource-intensive, and most clusters have spare capacity available for BestEffort pods. For completeness, adding a request would be useful to force the cluster autoscaler to make room for the operator (guaranteeing that some forward progress will be made) and limits would be useful to make issues like leaks more apparent.

Proposal

Add a small request/limit for the pod based on current usage. In our cluster, the operator is extremely lightweight:

$ kubectl top pod -n spicedb-operator
NAME                                CPU(cores)   MEMORY(bytes)   
spicedb-operator-6cfc749745-2jdcr   2m           20Mi 

So something like this seems suitable:

    resources:
      requests:
        memory: "32Mi"
        cpu: "20m"
      limits:
        memory: "128Mi"
        cpu: "250m"

[ecordell]

I know we have limits now, but just wanted to address:

In our cluster, the operator is extremely lightweight

This is intentional, and is part of the reason that https://github.com/authzed/controller-idioms exists. It's too easy to write controllers with ballooning caches and lots of churn if you're not careful, we wanted to make it easier to write efficient operators.