EKS: Add Uploading Helm Charts to ECR Capability

[hakenmt]

Describe the feature

For fully private EKS clusters, helm charts for things like istio and the aws-load-balancer-controller need to placed somewhere like ECR. The Kubectl function already contains the helm binary for applying helm charts. It would be great if functionality could be added to push helm charts to ECR so that users don't need to create their own Lambda layer and function just to achieve that functionality.

Use Case

For fully private EKS clusters, not having to create and manage a separate function to do helm push would help speed up development time and remove some ambiguity about the best way to host helm charts for private clusters.

Proposed Solution

Since helm is already in the Kubectl function, add a new construct for pushing a helm chart to ECR. The helm chart would be in s3, get pulled to the Lambda function, then pushed to ECR using OCI.

Other Information

No response

Acknowledgements

• [X] I may be able to implement this feature request
• [ ] This feature might incur a breaking change

CDK version used

2.138.0

Environment details (OS name and version, etc.)

darwin

[pahud]

Are you requesting a feature to allow CDK to do helm push to private ECR repo? This is interesting. Can you share how would you implement this feature and what is the high-level API experience like?

[hakenmt]

Yes, would like to be able to do a helm push to a private ECR repo. Currently, I'm doing this:

Repository repo = new Repository(this, name + "HelmRepo", new RepositoryProps() {
                EmptyOnDelete = true,
                RemovalPolicy = RemovalPolicy.DESTROY,
                RepositoryName = name
            });

            CustomResource chart = new CustomResource(this, name + "HelmChart", new CustomResourceProps() {
                ServiceToken = function.FunctionArn,
                Properties = new Dictionary<string, object> {
                    { "Type", "Helm" },
                    { "Bucket", Fn.Ref("AssetsBucket") },
                    { "Key", Fn.Ref("AssetsBucketPrefix") + "helm/" + name + "-" + version + ".tgz" },
                    { "Repository", repo.RepositoryName }
                }
            });

And the Lambda function looks generally like this:

request_type = event["RequestType"]

    if request_type == "Create" or request_type == "Update":

        try:
            type = event["ResourceProperties"]["Type"]
            bucket = event["ResourceProperties"]["Bucket"]
            key = event["ResourceProperties"]["Key"]
            repo_name = event["ResourceProperties"]["Repository"]

            if type == "Helm":

                response = ecr_client.get_authorization_token(registryIds = [ os.environ.get("AWS_ACCOUNT_ID") ])
                username, password = base64.b64decode(response["authorizationData"][0]["authorizationToken"]).decode().split(":")
                endpoint = response["authorizationData"][0]["proxyEndpoint"]
                domain = urlparse(endpoint).netloc

                s3_client.download_file(bucket, key, "/tmp/" + key.split("/")[-1])

                output = subprocess.check_output(["helm", "registry", "login", "--username", username, "--password", password, domain], stderr=subprocess.STDOUT, cwd="/tmp")
                print(output)
                output = subprocess.check_output(["helm", "push", "/tmp/" + key.split("/")[-1], "oci://" + domain ], stderr=subprocess.STDOUT, cwd="/tmp")
                print(output)
                images = ecr_client.describe_images(repositoryName = repo_name)
                print(json.dumps(images, default = str))

                if "imageDetails" in images and len(images["imageDetails"]) > 0 and "imageTags" in images["imageDetails"][0]:
                    responseData["Tags"] = images["imageDetails"][0]["imageTags"]

                responseData["RepositoryName"] = repo_name

                cfnresponse.send(event, context, cfnresponse.SUCCESS, responseData)

Turning this into a construct would basically just convert the custom resource into a type like ECRHelmChart and provide the same input parameters, the bucket and key where the *.tgz file is located (or perhaps a URL where it could be copied from, or maybe more generally an asset that could either be a locally created helm chart, copied from an existing helm chart repo, or a file already downloaded).

ECRHelmChart chart = new ECRHelmChart(this, "MyHelmChart", new ECRHelmChartProps() {
   Asset = Asset.FromURL("https://..."),
   Repository = repo.RepositoryName
});

[pahud]

Yes this sounds very promising! Please help us prioritize with 👍 . We welcome all pull requests from the community as well. For more information, refer to the CDK Developer Guide and Contributing Guidelines.