[aws-eks] Managed NodeGroup disk encryption

[DarkmatterVale]

Encrypt EBS volumes backing Managed NodeGroup EC2 instances.

Use Case

We want to enforce security best practices when using EKS Managed NodeGroups.

Proposed Solution

Add a boolean parameter "diskEncrypted" in managed-nodegroup.

Requires EKS NodeGroup CFN to add a boolean encryption parameter as well.

Other

• AWS EKS NodeGroup CFN specification

• [ ] :wave: I may be able to implement this feature request

• [ ] :warning: This feature might incur a breaking change

---

This is a :rocket: Feature Request

[eladb]

We won't be able to support this until CFN support is added (copy @tabern)

[alanprot]

I guess we can achieve the EBS encryption using LaunchTemplate?

https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-ec2-launchtemplate-launchtemplatedata.html

https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-ec2-launchtemplate-blockdevicemapping-ebs.html

But seems that CDK also does not allow to specify a launch template for the node group.

[iliapolo]

@alanprot Actually launch template support was just added: https://github.com/aws/aws-cdk/pull/9881

Should be available in the next release.

[shrivastavshubham34]

@iliapolo I don't see the encryption option below, is this a work in progress? https://docs.aws.amazon.com/cdk/api/latest/docs/@aws-cdk_aws-ec2.EbsDeviceProps.html

[iliapolo]

@shrivastavshubham34 I was referring to the addition of launch template support in EKS. You can achieve the encryption by specifying it in the launch template using the ec2.CfnLaunchTemplate construct.

I'm not sure exactly what the EbsDeviceProps you link are referring to in this context.

[shrivastavshubham34]

Thanks, ec2.LaunchTemplate has argument called blockDevices which had BlockDeviceVolume, but i'll try CfnLaunchTemplate

[shrivastavshubham34]

@iliapolo I keep getting "Instance types must be specified within the launch template". I already have a construct specifying this, but it won't accept. Is there a workaround?

        lt = ec2.CfnLaunchTemplate(self, "LaunchTemplate",
            launch_template_data={
                # "instance_type": CFG['eks']['ngIntel']['instance_type'],
                "block_device_mappings": {
                    "ebs": {
                        "encrypted": "true"
                    }
                }
            }
        )

        ng1 = eks_cluster.add_nodegroup_capacity('ng'+CFG['eks']['ngIntel']['type']+CFG['environment_name'],
        ami_type=eks.NodegroupAmiType.AL2_X86_64, # AL2_ARM_64, change for ARM
        launch_template_spec={
            "id": lt.ref,
            "version": lt.attr_latest_version_number
        },
        remote_access=eks.NodegroupRemoteAccess(ssh_key_name=CFG['eks']['ngIntel']['key_pair'],source_security_groups=[remote_access_sg]), 

        instance_type=ec2.InstanceType(CFG['eks']['ngIntel']['instance_type']),
        subnets=ec2.SubnetSelection(subnets=[s for s in vpc.private_subnets if s.subnet_id in CFG['subnets']['private']]),
        min_size=CFG['eks']['ngIntel']['min_size'],
        max_size=CFG['eks']['ngIntel']['max_size']
        )

[iliapolo]

@shrivastavshubham34 Are you getting this at deploy time? Can you share the node group section of the synthesized CloudFormation template?

[shrivastavshubham34]

Sure, @iliapolo Current nodegroup configuration

  EKSClusterdevNodegroupnginteldevC0164185:
    Type: AWS::EKS::Nodegroup
    Properties:
      ClusterName:
        Ref: EKSClusterdev8A9BE0DD
      NodeRole:
        Fn::GetAtt:
          - EKSClusterdevNodegroupnginteldevNodeGroupRoleB0F8FD70
          - Arn
      Subnets:
        - subnet-xxxxxxxxxxxxxxxxxxx
        - subnet-xxxxxxxxxxxxxxxxxxx
      AmiType: AL2_x86_64
      ForceUpdateEnabled: true
      InstanceTypes:
        - m5.large
      RemoteAccess:
        Ec2SshKey: key1
        SourceSecurityGroups:
          - sg-xxxxxxxxxxxxxxxxx
      ScalingConfig:
        DesiredSize: 3
        MaxSize: 5
        MinSize: 3
    Metadata:
      aws:cdk:path: microeks-cdk-dev/EKS_Cluster_dev/Nodegroupnginteldev/Resource

Nodegroup configuration after I add LaunchTemplate:

  EKSClusterdevNodegroupnginteldevC0164185:
    Type: AWS::EKS::Nodegroup
    Properties:
      ClusterName:
        Ref: EKSClusterdev8A9BE0DD
      NodeRole:
        Fn::GetAtt:
          - EKSClusterdevNodegroupnginteldevNodeGroupRoleB0F8FD70
          - Arn
      Subnets:
        - subnet-xxxxxxxxxxxxxxxxxxx
        - subnet-xxxxxxxxxxxxxxxxxxx
      AmiType: AL2_x86_64
      ForceUpdateEnabled: true
      LaunchTemplate:
        Id:
          Ref: LaunchTemplate
        Version:
          Fn::GetAtt:
            - LaunchTemplate
            - LatestVersionNumber
      RemoteAccess:
        Ec2SshKey: key1
        SourceSecurityGroups:
          - sg-xxxxxxxxxxxxxxxxxxx
      ScalingConfig:
        DesiredSize: 3
        MaxSize: 5
        MinSize: 3
    Metadata:
      aws:cdk:path: microeks-cdk-dev/EKS_Cluster_dev/Nodegroupnginteldev/Resource

[iliapolo]

@shrivastavshubham34 What stands out from the template after you add the launch template is that its missing the instance types declaration from before:

InstanceTypes:
  - m5.large

This means that instance types are not defined nor in the node group config, nor in the launch template. So the error comes up. Are you by chance removing the instance_type property from the config when you add the launch template?

Also note that your CfnLaunchTemplate is not correctly configured. It should use strong types, not dictionaries:

        lt = ec2.CfnLaunchTemplate(self, "LaunchTemplate",
            launch_template_data=ec2.CfnLaunchTemplate.LaunchTemplateDataProperty(
              block_device_mappings=[
                ec2.CfnLaunchTemplate.BlockDeviceMappingProperty(
                  ebs=ec2.CfnLaunchTemplate.EbsProperty(
                    encrypted=True))],
              instance_type="m5.large"))

If you have any additional questions/problems i'll ask you to please open a dedicated issue so we keep this issue clean - we have already diverged quite a bit :)

Thanks

[shrivastavshubham34]

@iliapolo sorry for the delayed response, I was able to fix the issue. And yes I did not remove the instance_type property from the config, it just doesn't accept it. I had to add the instance_type property to launch the template and recreate the node group to make it work, but it's okay since it spins up a new one before deleting the old NG.

        ng_encrypted_lt = ec2.CfnLaunchTemplate(self, "NGEncryptLaunchTemplate",
            launch_template_data=ec2.CfnLaunchTemplate.LaunchTemplateDataProperty(
                instance_type=CFG['eks']['ngIntel']['instance_type'],
                key_name=CFG['eks']['ngIntel']['key_pair'],
                block_device_mappings=[ec2.CfnLaunchTemplate.BlockDeviceMappingProperty(
                    device_name="/dev/xvda",
                    ebs=ec2.CfnLaunchTemplate.EbsProperty(
                        volume_type="gp2",
                        volume_size=20,
                        encrypted=True
                    )
                )]
            )
        )

[maeghan-porter]

My use case where I ran in to needing this is I have an existing cluster I am importing and when I create a nodegroup for it, without a launch template, the nodes come up great and connect to the cluster just fine, but they are lacking block device encryption, which we want for security hygiene purposes. However if I add a launch template where I'm just defining the block device and setting it to encrypted, the nodes that come up in that format can no longer connect to the cluster. I'm really not sure why this is. But as a result I cannot use the launch template and so cannot set the block device to be encrypted. Given that, I figure if I could set the block devices to be encrypted via the nodegroup definition, in theory, whatever is happening in the background that gives my nodegroups access to the cluster should work.

[pahud]

Still relevant. I will look into this and see if there's anything we could do to work around this.

[iliapolo]

It sounds the solution for this is 3 fold:

1. Add support for ebs encryption in the ec2.LaunchTemplate construct.
2. Make the Nodegroup construct use the ec2.LaunchTemplate by default.
3. Possibly expose an encrypted property on the Nodegroup construct to make it easier to create default encrypted volumes.

Putting this here for anyone looking interested in contributing. We still haven't put this on our roadmap. Stay tuned.

[pahud]

Yep we should support ebs encryption for the ec2.LaunchTemplate construct.

https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-ec2-launchtemplate-ebs.html#cfn-ec2-launchtemplate-ebs-encrypted

related to https://github.com/aws/aws-cdk/issues/6459