ec2_instance_credentials requires role arn

[yalwan-sage]

I was looking here: https://github.com/JuliaCloud/AWS.jl/blob/master/src/AWSCredentials.jl#L259 because I was troubleshooting an issue why the credentials were not automatically obtained. Our systems block this call: ec2_instance_metadata("/latest/meta-data/iam/info")

Further to that, even if it didn't throw (because the block 404s), but instead returned nothing, I wouldn't be able to get credentials because at line 256 we bail early if there isn't any info. As best I can tell it is used to fill in the ARN field, but I can see other constructors where the ARN field is permitted to be empty.

Is it possible to relax this?

[mattBrzezinski]

EC2 Instance Credentials is the final attempt to get credentials. I would suggest looking at your environment variables or ~/.aws/config or ~/.aws/credentials files to double check that you're setting them properly if you are failing to get them.

If the result from meta-data/iam/info is nothing we exit because you cannot make a connection to this endpoint. This is only accessible when you are on an EC2 instance. It this result fails, the subsequent ones for name and creds will fail also.

If you are on an EC2 instance, you will have an IAM Role attached to it, so we want to return back this information. It's optional elsewhere because other forms of credentials may no necessarily be roles attached to an instance, or assumed roles.

[yalwan-sage]

Thanks for quick response!

Here is some context: The runtime is a kubernetes pod in aws with KIAM so it looks like it has a metadata store (this is responsible for the IMDS filter) Aws cli and boto3 in the same environment are able to obtain credentials

The call i mentioned above throws a 404 so whole process halts, but I was noting that if it gave a 401 forbidden instead (which wouldn't bubble up as far as i can tell) there would be nothing and we would bail, but apart from ARN we could still fill the struct with credentials. I wrote an own credentials provider which is a subset of the ec2 credentials call (fills the arn with empty string) and it worked fine, hence why I was asking if this requirement could be relaxed.

[mattBrzezinski]

Is there a reason why you're blocking the call to some metadata endpoints but not others? I suppose we can modify this line to check if it haskey("instanceArn") and use it, otherwise default to an empty string.

This shouldn't create any issues going forward.

[yalwan-sage]

I wasn't able to find that there was a good reason for it (yet) but I raised this issue because it seemed to me like if other tools can work under the conditions then this probably could/should too, would you agree?

[yalwan-sage]

I just want to point out as well that the metadata filtering raises a 404 which currently bubbles up, so currently we never get that far to the code to check.

[mattBrzezinski]

I just want to point out as well that the metadata filtering raises a 404 which currently bubbles up, so currently we never get that far to the code to check.

Definitely makes sense. Feel free to make a PR for this, and I can review it. Otherwise I can get around to this some time later.

[yalwan-sage]

I wanted to start on this but also wanted to get a feedback:

I'm thinking of modifying the ec2_instance_metadata function like so:

function ec2_instance_metadata(path::AbstractString, ignored_codes::Tuple{Vararg{<:Integer}}=())
    uri = HTTP.URI(; scheme="http", host="169.254.169.254", path=path)
    request = try
        Mocking.@mock HTTP.request("GET", uri; connect_timeout=1)
    catch e
        if e isa HTTP.ConnectionPool.ConnectTimeout
            nothing
        elseif e isa HTTP.StatusError
            if e.status in ignored_codes
                nothing
            else
                rethrow()
            end
        end
    end

    return request !== nothing ? String(request.body) : nothing
end

and then for the info retrieval request we can ask to ignore 404 specifically, and then do the remainder of what was needed (i.e. still return credentials without RoleArn if we don't get this)

Is this acceptable?