Use Quay registry by default instead of Docker Hub for deploying Calico

Exagone313 commented 1 year ago

Docker Hub is sunsetting free organisations next month. Calico is currently deployed using Docker Hub. While Tigera may choose to pay a subscription for the calico organisation, relying on public images on Docker Hub may be seen as a potential future failure, as these images may disappear unexpectedly.

Expected Behavior

Calico is deployed using the quay.io (Quay) registry when the spec.registry key of Installation custom resource is not declared.

Current Behavior

Calico is deployed using the docker.io (Docker Hub) registry when the spec.registry key of Installation custom resource is not declared.

Possible Solution

Patch CalicoRegistry https://github.com/tigera/operator/blob/74bdfe1e74d863961dedf2e1cb4d0d795e2dd1f8/pkg/components/images.go#L18

Steps to Reproduce (for bugs)

N/A

Context

Your Environment

N/A

onedr0p commented 1 year ago

Ref: https://github.com/docker/hub-feedback/issues/2314

danudey commented 1 year ago

Docker wrote a blog post to clarify their terrible announcement: https://www.docker.com/blog/we-apologize-we-did-a-terrible-job-announcing-the-end-of-docker-free-teams/

ceastman-r7 commented 1 year ago

I have installed tigera-operator with the following but still find some pods with images from docker.io:

installation:
  kubernetesProvider: EKS
  registry: quay.io/

so far:

kubectl get ds -o yaml -n calico-system csi-node-driver | grep -i 'image:' | sort | uniq
        image: docker.io/calico/csi:v3.24.1
        image: docker.io/calico/node-driver-registrar:v3.24.1

and

kubectl get deploy -o yaml -n calico-system calico-kube-controllers |grep -i 'image:' | sort | uniq
        image: docker.io/calico/kube-controllers:v3.24.1

Exagone313 commented 1 year ago

I have installed tigera-operator with the following but still find some pods with images from docker.io:

Your configuration may be wrong, can you provide your whole Installation definition? I don't have this issue.

ceastman-r7 commented 1 year ago

That is my whole values.yaml that I pass to to the tigera-operator helm chart. Mostly all the calico images are coming from quay.io, just not the ones I listed above.

ceastman-r7 commented 1 year ago

sorry this must be a cluster issue on my end, i checked another cluster and its fine.

tibeer commented 1 year ago

I have another point to do this change: docker.io is still not IPv6 capable. Yes, there is registry.ipv6.docker.com, but in my Scenario, I try to deploy calico on an IPv6 only machine. The operator keeps altering back the deployments and daemonsets to use docker.io which results in an Image Pull Error. Or maybe you guys can provide a config map for the operator and the registry(ies) to use for calico images?

tmjd commented 1 year ago

Or maybe you guys can provide a config map for the operator and the registry(ies) to use for calico images?

@tibeer have you tried configuring the registry field in the installation resource? I would think that might address the issue for an IPv6 registry.

Is quay.io IPv6 capable and would fix this issue for you without specifying any additional configuration?

tibeer commented 1 year ago

@tmjd this works! Is there an easy option to set this information via helm? I did not find a reference in the chart values.

tmjd commented 1 year ago

@tibeer Helm configuration has an embedded installation spec. That is what the installation field in the chart values equates to. So anything you could put in the Installation resource spec you can put in helm, just make sure it is correctly nested as it would be in the the Installation spec. Or at least that is my understanding.

tibeer commented 1 year ago

@tmjd after a bit of trying I figured it out:

helm install calico ./tigera-operator --namespace tigera-operator --create-namespace --set installation.registry="quay.io"

This was not easy to find, as the documentation references it as "InstallationSpec", so initially thought InstallationSpec.registry="quay.io" was the right value.

Oh, and another thing: You have to copy the helm-chart manually, as

helm repo add projectcalico https://docs.tigera.io/calico/charts

isn't working on IPv6 only either, since it seems to be hosted on github as well :/

But I got it working for me now, thanks a lot!

jodem commented 1 year ago

A good rules a thumb from security perspective IMHO is to aways duplicate the chart used images to your own registry. It's painful, it takes times to maintain but reduce risk of availability if original registry goes down (or has rate limits...), and system integrity since the source image could get compromised afterward (with a malware for instance).

tibeer commented 8 months ago

@Exagone313 @tmjd since the issue is now almost a year old and considering that the calico images are still pulled from dockerhub and nothing bad happened in that time, can we close this issue?

Exagone313 commented 8 months ago

I think it should be more straightforward to use Quay instead of Docker Hub. Currently, if I try to install Calico, I would not even know that images are also pushed on Quay.

As far as I know, Docker Hub still has some of these quota built-in unlike Quay. Also Docker Hub randomly fails (EOF) and does not support IPv6 from what is told above.

What is the advantage of keeping Docker Hub the default?

tigera / operator