RHEL images can't authenticate with Yum repos

[tomstockton]

I want to use Aminator to create RHEL based AMIs. Aminating the image works as expected and I can boot the image, however when I attempt to use Yum I get errors such as:

https://rhui2-cds02.eu-west-1.aws.ce.redhat.com/pulp/repos//rhui-client-config/rhel/server/6/x86_64/os/repodata/repomd.xml: [Errno 14] PYCURL ERROR 22 - "The requested URL returned error: 401 Authorization Required"

I've researched this issue and it is fairly common on the AWS forums. The outcome seems to be that creating AMIs from Snapshots do not result in the AMI having the appropriate 'billingProducts' set. https://forums.aws.amazon.com/thread.jspa?messageID=440277&#440277

I can confirm that this is the case when I query the metadata on my aminated image.

# curl -s http://169.254.169.254/latest/dynamic/instance-identity/document | grep billingProducts
  "billingProducts" : null,

Vs the same query on the base image that aminator used.

# curl -s http://169.254.169.254/latest/dynamic/instance-identity/document | grep billingProducts
  "billingProducts" : [ "bp-6fa54006" ],

Is anybody aminating RHEL images? Is there a workaround that I'm missing?

Thanks

Tom

[coryb]

Hey Tom, I am not familiar with people using RHEL directly with aminator. We have used Centos, and that obviously wont have the same RHEL registration issues.

Perhaps there needs to be an extra parameter set on the ami registration. Maybe you could run this command (with the python based aws cli):

aws ec2 describe-images --image-id $ami

for both the AMI with the correct billingProduct and the AMI that aminator generated?

[tomstockton]

Hi Cory,

Thanks for the response, please see the requested output below. Unfortunately it doesn't seem to list any billingProduct attributes so I'm not sure how helpful this is.

AMI with correct billingProduct

{
    "Images": [
        {
            "VirtualizationType": "hvm",
            "Name": "****_base_rhel_65_hvm_20150401_1",
            "Hypervisor": "xen",
            "ImageId": "ami-****",
            "State": "available",
            "BlockDeviceMappings": [
                {
                    "DeviceName": "/dev/sda1",
                    "Ebs": {
                        "DeleteOnTermination": true,
                        "SnapshotId": "snap-****",
                        "VolumeSize": 10,
                        "VolumeType": "gp2"
                    }
                }
            ],
            "Architecture": "x86_64",
            "ImageLocation": "****/****_base_rhel_65_hvm_20150401_1",
            "RootDeviceType": "ebs",
            "OwnerId": "****",
            "RootDeviceName": "/dev/sda1",
            "Public": false,
            "ImageType": "machine",
            "Description": "****_base_rhel_65_hvm_20150401_1"
        }
    ]
}

Aminated image without billingProduct set

{
    "Images": [
        {
            "VirtualizationType": "hvm",
            "Name": "****_aminator_2015.04.10.14.34-ebs",
            "Tags": [
                {
                    "Value": "-****-201504101435",
                    "Key": "appversion"
                },
                {
                    "Value": "devops@****.net",
                    "Key": "creator"
                },
                {
                    "Value": null,
                    "Key": "base_ami_version"
                },
                {
                    "Value": "2015-04-10 13:41:08 UTC",
                    "Key": "creation_time"
                }
            ],
            "Hypervisor": "xen",
            "ImageId": "ami-****",
            "State": "available",
            "BlockDeviceMappings": [
                {
                    "DeviceName": "/dev/sda1",
                    "Ebs": {
                        "DeleteOnTermination": true,
                        "SnapshotId": "snap-****",
                        "VolumeSize": 10,
                        "VolumeType": "standard"
                    }
                },
                {
                    "DeviceName": "/dev/sdb",
                    "VirtualName": "ephemeral0"
                },
                {
                    "DeviceName": "/dev/sdc",
                    "VirtualName": "ephemeral1"
                },
                {
                    "DeviceName": "/dev/sdd",
                    "VirtualName": "ephemeral2"
                },
                {
                    "DeviceName": "/dev/sde",
                    "VirtualName": "ephemeral3"
                }
            ],
            "Architecture": "x86_64",
            "ImageLocation": "****/****_aminator_2015.04.10.14.34-ebs",
            "RootDeviceType": "ebs",
            "OwnerId": "****",
            "RootDeviceName": "/dev/sda1",
            "Public": false,
            "ImageType": "machine",
            "Description": "name=, arch=x86_64, ancestor_name=****_base_rhel_65_hvm_20150401_1, ancestor_id=ami-****, ancestor_version="
        }
    ]
}

Hope this helps, please let me know if I can provide any more information.

Thanks

Tom

[coryb]

Hey Tom, yeah it appears the data is with the image attributes, not the image directly. I found this document from AWS: http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/paid-amis.html

From reading the doc it looks like you need to do something like:

aws ec2 modify-image-attribute --image-id ami_id --product-codes "product_code"

That is something I think you would have to run every time after the amination process finished. Or if you are so inclined we would take a pull request if you wanted to make aminator do this with a new --product-code option.

-Cory

[tomstockton]

Hi Cory,

This looked promising initially, however after further investigation I'm not sure that 'product code' is the same thing as the 'billing code'. From what I can see the RHEL base image does not contain a 'product code'.

GET http://169.254.169.254/latest/meta-data/product-codes

Running the above gives a 404

The same outcome can be seen in the AWS console when looking under 'AMIs' -> 'Public images' and searching for ami ' ami-04a00d73'. The 'Product Codes' column is empty.

I did try to run the command you suggested but received the following error - this suggests to me that product and billing codes are not the same thing!

$ aws ec2 modify-image-attribute --image-id ami-2d32545a --product-codes "bp-6fa54006"
A client error (InvalidParameterValue) occurred when calling the ModifyImageAttribute operation: Invalid value 'bp-6fa54006' for productCode.

I've posted a message on the AWS forums to clarify this - https://forums.aws.amazon.com/thread.jspa?threadID=121374

[stevenlattin]

Hello All, I had some rather frustrating time on this issue and thought I would add what I found to solve this problem in case the next person rolls onto it.

As stated above the "Billing code" is required in the instance details document. The process of creating a snapshot from a volume (per AWS process) strips this field. Any AMI's produced from this SS will likely have this issue present (I have repro'd about a dozen times). However, If you create an image from a redhat instance without stopping etc... (right-click on instance and chose image >> create image) you will get an AMI as a result. This path will preserve the billing code. In addition if you want an encrypted AMI then you can copy the AMI and click the checkbox for encryption. The billing code will be preserved through this method and YUM will work as expected.