AWS_USE_FIPS_ENDPOINT=true breaks aws autoscaling describe-auto-scaling-instances

[samholton]

Describe the bug

According to https://aws.amazon.com/compliance/fips/, autoscaling does not have a FIPS endpoint in commercial us-east-1. However, specifying that environment variable breaks the aws autoscaling describe-auto-scaling-instances. Is the CLI supposed to enable FIPS endpoints only if they exist when setting AWS_USE_FIPS_ENDPOINT=false? Or do I need to selectively set them for calls which have FIPS endpoints?

export INSTANCEID=<an instance id>

# working case with FIPS disabled
AWS_USE_FIPS_ENDPOINT=false aws autoscaling describe-auto-scaling-instances --instance-ids "$INSTANCEID"
<redacted json>

# not working when setting FIPS true
AWS_USE_FIPS_ENDPOINT=true aws autoscaling describe-auto-scaling-instances --instance-ids "$INSTANCEID"

Could not connect to the endpoint URL: "https://autoscaling-fips.us-east-1.amazonaws.com/"

Expected Behavior

Ideally fall back and use non-FIPS endpoint as it doesn't appear there is a FIPS autoscaling endpoint - returns same data as when making the call using AWS_USE_FIPS_ENDPOINT=false.

Current Behavior

AWS_USE_FIPS_ENDPOINT=true aws autoscaling describe-auto-scaling-instances --instance-ids "$INSTANCEID" --query 'AutoScalingInstances[].AutoScalingGroupName' --output text

Could not connect to the endpoint URL: "https://autoscaling-fips.us-east-1.amazonaws.com/"

Reproduction Steps

export INSTANCEID=<an instance id>
AWS_USE_FIPS_ENDPOINT=false aws autoscaling describe-auto-scaling-instances --instance-ids "$INSTANCEID"

Possible Solution

No response

Additional Information/Context

Running on an EC2 instance with instance profile gives same results as running locally with temporary credentials from STS. The role has permissions, setting AWS_USE_FIPS_ENDPOINT=false fixes the issue.

CLI version used

aws-cli/2.11.4 Python/3.11.2 Linux/3.10.0-1160.88.1.el7.x86_64 exe/x86_64.centos.7 prompt/off

Environment details (OS name and version, etc.)

CentOS Linux release 7.9.2009 (Core)

[tim-finnigan]

Hi @samholton thanks for reporting this issue. The endpoint ruleset that the CLI uses for the Auto Scaling service can be found here: https://github.com/boto/botocore/blob/develop/botocore/data/autoscaling/2011-01-01/endpoint-rule-set-1.json.

Based on that, I would expect the https://autoscaling-fips.us-east-1.amazonaws.com/ to resolve. I think this is a service-side issue that the Auto Scaling team needs to investigate further. I'm going to transfer this issue to our cross-SDK repository as other SDKs are affected and will update the issue when we have more information.

[tim-finnigan]

P86410350

[samholton]

@tim-finnigan is the documentation out of date then? Or maybe the endpoint ruleset is incorrect? According to https://aws.amazon.com/compliance/fips/ there is not a FIPS endpoint for autoscaling.

[tim-finnigan]

@tim-finnigan is the documentation out of date then? Or maybe the endpoint ruleset is incorrect? According to https://aws.amazon.com/compliance/fips/ there is not a FIPS endpoint for autoscaling.

This is another thing I'm checking in with the Auto Scaling team about. There is an internal process for service teams to register their endpoint configuration for display on that page, so if that FIPS endpoint is supported then it should be added there.

[tim-finnigan]

Checking in - reclassifying this as a feature request for autoscaling to support FIPS endpoints as it is not currently documented here: https://docs.aws.amazon.com/general/latest/gr/autoscaling_region.html

[samholton]

Ok, so intended behavior is failure when AWS_USE_FIPS_ENDPOINT is set to true and the service does not have a FIPS endpoint (rather than fallback). Makes sense.

[avivgold098]

This seemed like a wide issue in the Gov services which the doc and the endpoint generation have this mismatch. It's not clear how this issue close - https://github.com/aws/aws-sdk/issues/268

[avivgold098]

@RanVaknin - regrading https://github.com/aws/aws-sdk/issues/268 which is similar. Who in AWS is on it? Which service team?

[tim-finnigan]

Checking in again, thanks for your patience. After discussing this with more teams, here is my understanding of the issue: The FIPS page referenced earlier (https://aws.amazon.com/compliance/fips/) lists the valid FIPS endpoints by service. However, not every service follows the expected naming convention (which should be <service>-fips.<region>.<domain suffix>.) For example, the FIPS endpoints for EC2 Auto Scaling are: autoscaling.us-gov-east-1.amazonaws.com / autoscaling.us-gov-west-1.amazonaws.com. There are internal tracking tickets for services that don't follow the expected convention.

When AWS_USE_FIPS_ENDPOINT (documented here) is set to True the AWS SDKs endpoint ruleset routes to the standard naming convention. Until the individual service supports that convention, you can pass the actual FIPS endpoint URL manually in your SDK. There is no "fall-back" mechanism by design, as the correct endpoint routing has to be followed regardless of whether the service supports the expected FIPS endpoint. Hope that helps. Please let us know if you had any other questions related to this.

[github-actions[bot]]

This issue is now closed.

Comments on closed issues are hard for our team to see. If you need more assistance, please either tag a team member or open a new issue that references this one. If you wish to keep having a conversation with other community members under this issue feel free to do so.