Optimize image pull time/cost and ensure images are scanned

[mogul]

User Story

In order to minimize the time and cost for deploying EKS workloads, SSB operators want to use a pull-through cache on ECR Public to pull public images whenever possible (note Docker Official Images are available via ECR Public).

Acceptance Criteria

[ACs should be clearly demoable/verifiable whenever possible. Try specifying them using BDD.]

• [ ] GIVEN [a contextual precondition] \ [AND optionally another precondition] \ WHEN [a triggering event] happens \ THEN [a verifiable outcome] \ [AND optionally another verifiable outcome]

Background

[Any helpful contextual notes or links to artifacts/evidence, if needed]

Security Considerations (required)

[Any security concerns that might be implicated in the change. "None" is OK, just be explicit here!]

Sketch

• [ ] Configure pull-through rules
• [ ] Test using the cluster's private ECR registry for pulls, using the specified format
• [ ] See if we can automatically have our clusters translate Docker Hub pulls to private ECR pulls, eg using a custom mutating admission controller.
  • This one may do what we want
  • If not
  • [ ] switch our own URLs to the ECR Gallery URL for images we use, eg SolrCloud
  • [ ] figure out if we also have to re-publish our own Helm charts through ECR (yikes).

[mogul]

We should switch our use of helm charts to refer to ECR wherever possible. The charts in the EKS charts repository are all confirmed to only pull from ECR.

This tool is also useful to help identify other charts and images we're using that that are coming from Docker Hub.

[mogul]

dockerd can be configured with the location of a mirror for Docker Hub, which is how local dockerd instances can act as a pull-through cache.

There is a similar option for configuring containerd. EKS will soon switch to containerd as the default, and it's already available in EKS 1.21 using the stock AMIs.

Configuring the default registry to be our local ECR instance rather than Docker Hub will help ensure we are pulling everything through ECR.

[mogul]

There's a way to specify that MNGs should use containerd, but it's not clear how/if we can accomplish the same thing using the MNG launch template in the EKS module.

[mogul]

It's not yet possible to configure containerd to use a particular registry.