Remove CRDs from tigera-operator.yaml

caseydavenport commented 6 days ago

Description

Trying this out. It removes the CRDs from tigera-operator.yaml to reduce the size of the manifest.

Instead, it configures the operator pod itself to create these CRDs on startup.

The main consequences I can see are:

operator-init container in OCP will try to create resources before CRDs exist. May need to adjust the code.
users will need to wait for operator pod to be running before applying custom-resources.yaml

But I think this is probably worth trying out.

Related issues/PRs

Fixes https://github.com/projectcalico/calico/issues/9517

Requires: https://github.com/tigera/operator/pull/3610

Todos

[ ] Tests
[ ] Documentation
[ ] Release note

Release Note

To reduce manifest size, tigera-operator.yaml no longer includes CustomResourceDefinitions. Instead, the tigera-operator pod will deploy these resources on startup.

Reminder for the reviewer

Make sure that this PR has the correct labels and milestone set.

Every PR needs one docs-* label.

docs-pr-required: This change requires a change to the documentation that has not been completed yet.
docs-completed: This change has all necessary documentation completed.
docs-not-required: This change has no user-facing impact and requires no docs.

Every PR needs one release-note-* label.

release-note-required: This PR has user-facing changes. Most PRs should have this label.
release-note-not-required: This PR has no user-facing changes.

Other optional labels:

cherry-pick-candidate: This PR should be cherry-picked to an earlier release. For bug fixes only.
needs-operator-pr: This PR is related to install and requires a corresponding change to the operator.

caseydavenport commented 6 days ago

Actually looking at it, I think this doesn't impact OCP since we install CRDs as distinct manifests in that env.

caseydavenport commented 6 days ago

Suspect there is some more operator work required before merging this, but not too much.

lwr20 commented 4 days ago

users will need to wait for operator pod to be running before applying custom-resources.yaml

If we continue to provide the CRDs manifest, presumably users can continue to deploy them if they want to (and therefore apply custom-resources.yaml before operator)

Is there mileage in just including the CRDs required for custom-resources in the operator yaml? Or are they all needed?

caseydavenport commented 3 days ago

@lwr20 I think there is mileage in that, although I'd need to think about what work is required on our end to make that happen. Not obvious how we get our generation code to do that nicely.

There's a part of me that says "Let's just roll this out and see how it fares" - it's still possible that even including our operator CRDs can exceed maximum sizes in some cases (if not now, then eventually)

coutinhop commented 3 days ago

LGTM, one question about manifests/ocp/policy.networking.k8s.io_adminnetworkpolicies.yaml: that's not a calico CRD per se, but it might be nice to have the operator take care of it and to remove the yaml from here as well, what do you think?

projectcalico / calico