search

googleapis / google-auth-library-java

Open source Auth client library for Java
https://developers.google.com/identity
BSD 3-Clause "New" or "Revised" License
411 stars 225 forks source link

Workload identity federation permission boundary problem with AWS ECS #794

Closed ColeSiegelTR closed 2 years ago

ColeSiegelTR commented 2 years ago

All containers running in an AWS ECS Cluster have permissions to make Google Cloud API calls as opposed to just the one required task.

To configure the workload identity federation for making Google Cloud API calls from AWS, we bind an AWS role ARN to a Google Service account. And the Google cloud SDK uses the AWS metadata endpoint to retrieve the current active role. When we run this inside an AWS ECS task, the metadata endpoint returns the underlying EC2 node role instead of the Task role. This forces the user to bind the underlying node role, thereby providing expanding the permission boundary more than desired. This also hinders least privileged access - meaning provide Tasks with their own level of permissions.

There is some inconsistency in the AWS metadata endpoint (when running inside EC2 or Lambda vs ECS) and the way the Google Cloud SDK retrieves the active AWS role.

https://github.com/googleapis/google-auth-library-java/blob/1149581e63267e3553c74ba2114d849c5b24f27b/oauth2_http/java/com/google/auth/oauth2/AwsCredentials.java#L107

chanseokoh commented 2 years ago

@lsirac

bojeil-google commented 2 years ago

Hey @ColeSiegelTR can you clarify what the inconsistency is here?

ColeSiegelTR commented 2 years ago

Hi @bojeil-google, the metadata endpoint returns the underlying EC2 node role instead of the current active Task role, which forces us to provide a larger scope of permissions than required when configuring workload identity federation with GCP. I would expect that when binding an AWS role with GCP workload identity, we can specify the AWS role of the task rather than the AWS role of the node.

I am not sure the resolution, maybe AWS has a different metadata endpoint which return a more specific role, rather than this underlying node role which your library is using?

bojeil-google commented 2 years ago

This is what is made available by AWS. We need the ability to generate signed requests for the desired AWS role. We can either do it via AWS metadata server directly or via AWS environment variables. Basically we use the AWS security credentials to create the signed requests and verify them on our end. You can't just specify a role explicitly as we cannot trust that. We can only trust signed requests.

Maybe you should reach out to AWS support about this. Perhaps they have some other endpoint or ability to generate signed requests using the current active task role.

ColeSiegelTR commented 2 years ago

Hi @bojeil-google

I did some research and they do have an endpoint for receiving ECS task metadata as opposed to the fetching the underlying EC2 node's metadata.

https://docs.aws.amazon.com/AmazonECS/latest/developerguide/task-metadata-endpoint.html

I think this library should be using the task metadata endpoint 169.254.170.2, but it is instead using the underlying EC2 metadata endpoint 169.254.169.254. This means we need to expand permissions more broadly than we would've liked.

Let me know if it makes sense or if you need more clarification.

bojeil-google commented 2 years ago

I think the library is doing as expected. We are not going to change the existing behavior. It will break all of our customers. However, we will do more research on supporting this use case (and what the ECS task metadata provides) and treat it as a feature request. Customers may end up choosing which one to use.

Also thanks @ColeSiegelTR for bringing this to our attention. We haven't thought of this use case until now.

ColeSiegelTR commented 2 years ago

Hey @bojeil-google

Thanks for your response, that makes sense. I am quite new to this library myself so I need to dig in a bit more on our use case and also understand how/where these metadata endpoints are retrieved by the library. We appreciate you taking a look and I will provide more information if I have any.

lesv commented 2 years ago

I'm going to close this, feel free to reopen or ask us to if you have additional questions.

ColeSiegelTR commented 2 years ago

Hey @lesv and @bojeil-google

Is there any ticket we can track for the feature request? I took a closer look at the code and it doesn't look there is anything we can do on our end to better support this use case. Thanks a lot.