chime / terraform-aws-alternat

High availability implementation of AWS NAT instances.
MIT License
1.06k stars 66 forks source link

Root EBS - /dev/xvda for Arm images #56

Closed Halama closed 1 year ago

Halama commented 1 year ago

Issue

When I am using ARM images alternat provisions another EBS as /dev/sda1 and root EBS is still 8GB unencrypted. I assume this is not intentional and there should be only one root EBS with encryption enabled:

image

Fix

I've changed the configured block device mapping to /dev/xvda which is root volume for ARM images https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/device_naming.html?icmpid=docs_ec2_console#device-name-limits

aws ec2 describe-images --image-ids ami-0cd7323ab3e63805f --region us-east-1
{
    "Images": [
        {
            "Architecture": "arm64",
            "CreationDate": "2023-02-09T11:53:44.000Z",
            "ImageId": "ami-0cd7323ab3e63805f",
            "ImageLocation": "amazon/amzn2-ami-kernel-5.10-hvm-2.0.20230207.0-arm64-gp2",
            "ImageType": "machine",
            "Public": true,
            "OwnerId": "137112412989",
            "PlatformDetails": "Linux/UNIX",
            "UsageOperation": "RunInstances",
            "State": "available",
            "BlockDeviceMappings": [
                {
                    "DeviceName": "/dev/xvda",
                    "Ebs": {
                        "DeleteOnTermination": true,
                        "SnapshotId": "snap-04cb0f411fe7a3137",
                        "VolumeSize": 8,
                        "VolumeType": "gp2",
                        "Encrypted": false
                    }
                }
            ],
            "Description": "Amazon Linux 2 LTS Arm64 Kernel 5.10 AMI 2.0.20230207.0 arm64 HVM gp2",
            "EnaSupport": true,
            "Hypervisor": "xen",
            "ImageOwnerAlias": "amazon",
            "Name": "amzn2-ami-kernel-5.10-hvm-2.0.20230207.0-arm64-gp2",
            "RootDeviceName": "/dev/xvda",
            "RootDeviceType": "ebs",
            "SriovNetSupport": "simple",
            "VirtualizationType": "hvm",
            "DeprecationTime": "2025-02-09T11:53:44.000Z"
        }
    ]
}

After the fix:

image
bwhaley commented 1 year ago

Oh, I'm also wondering: do you know what the impact of this would be for non-graviton instance types? Will they be fine with xvda?

Halama commented 1 year ago

@bwhaley I've tried few x86 images and it was also returning /dev/xvda.

 aws ec2 describe-images --image-ids ami-05b5badc2f7ddd88d --region us-east-1
{
    "Images": [
        {
            "Architecture": "x86_64",
            "CreationDate": "2023-02-24T09:54:00.000Z",
            "ImageId": "ami-05b5badc2f7ddd88d",
            "ImageLocation": "amazon/amzn2-ami-hvm-2.0.20230221.0-x86_64-gp2",
            "ImageType": "machine",
            "Public": true,
            "OwnerId": "137112412989",
            "PlatformDetails": "Linux/UNIX",
            "UsageOperation": "RunInstances",
            "State": "available",
            "BlockDeviceMappings": [
                {
                    "DeviceName": "/dev/xvda",
                    "Ebs": {
                        "DeleteOnTermination": true,
                        "SnapshotId": "snap-07d0aaf53127ea133",
                        "VolumeSize": 8,
                        "VolumeType": "gp2",
                        "Encrypted": false
                    }
                }
            ],
            "Description": "Amazon Linux 2 AMI 2.0.20230221.0 x86_64 HVM gp2",
            "EnaSupport": true,
            "Hypervisor": "xen",
            "ImageOwnerAlias": "amazon",
            "Name": "amzn2-ami-hvm-2.0.20230221.0-x86_64-gp2",
            "RootDeviceName": "/dev/xvda",
            "RootDeviceType": "ebs",
            "SriovNetSupport": "simple",
            "VirtualizationType": "hvm",
            "DeprecationTime": "2025-02-24T09:54:00.000Z"
        }
    ]
}

Another approach could be fetch device name from AMI data resource https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/ami Like this:

block_device_mappings {
    device_name = data.aws_ami.amazon_linux_2.block_device_mappings[0].device_name

    ebs {
      volume_size = 80
      encrypted   = true
    }
  }

But it should also handle situation when the custom ami is provided https://github.com/1debit/alternat/blob/2d2f0c6fee0f7fdd254355986e568ff0be82cdde/modules/terraform-aws-alternat/main.tf#L206

So maybe it would be better to be able to set block_device_mappings using variable, similar to https://github.com/terraform-aws-modules/terraform-aws-eks/blob/c7565e265e23cbc622041fc153193f490cfe948f/modules/self-managed-node-group/variables.tf#LL135-L139 ?

bwhaley commented 1 year ago

Great input, thanks. I opened a new PR with a slightly different approach. In that PR, if the volumes are permitted, we just use the AMI default (which is an 8GB gp2 volume on the Amazon Linux 2 AMI). What do you think?

Halama commented 1 year ago

I think https://github.com/1debit/alternat/pull/57 is a good approach and we'll use it this way when it'll be released. Thanks, I will now close this PR in favor of https://github.com/1debit/alternat/pull/57