Open ricardo8990 opened 3 months ago
I added the ENVOY_LOG_LEVEL
to DEBUG
and found this logs:
[2024-06-27 03:40:49.620][22][debug][aws] [source/extensions/common/aws/credentials_provider_impl.cc:67] Getting AWS credentials from the environment
[2024-06-27 03:40:49.620][22][debug][aws] [source/extensions/common/aws/credentials_provider_impl.cc:288] Getting AWS credentials from the task role at URI: http://169.254.170.23/v1/credentials
[2024-06-27 03:40:49.621][22][debug][misc] [source/extensions/common/aws/utility.cc:300] Could not fetch AWS metadata: HTTP response code said error
[2024-06-27 03:40:50.281][17][debug][main] [source/server/server.cc:263] flushing stats
[2024-06-27 03:40:50.281][17][debug][main] [source/server/server.cc:273] Envoy is not fully initialized, skipping histogram merge and flushing stats
[2024-06-27 03:40:50.622][22][debug][misc] [source/extensions/common/aws/utility.cc:300] Could not fetch AWS metadata: HTTP response code said error
[2024-06-27 03:40:51.623][22][debug][misc] [source/extensions/common/aws/utility.cc:300] Could not fetch AWS metadata: HTTP response code said error
[2024-06-27 03:40:52.624][22][debug][misc] [source/extensions/common/aws/utility.cc:300] Could not fetch AWS metadata: HTTP response code said error
[2024-06-27 03:40:53.624][22][error][aws] [source/extensions/common/aws/credentials_provider_impl.cc:302] Could not load AWS credentials document from the task role
[2024-06-27 03:40:53.625][22][debug][aws] [source/extensions/common/aws/credentials_provider_impl.cc:442] No AWS credentials found, using anonymous credentials
[2024-06-27 03:40:53.627][17][debug][grpc] [source/common/grpc/google_async_client_impl.cc:379] Finish with grpc-status code 16
[2024-06-27 03:40:53.627][17][debug][grpc] [source/common/grpc/google_async_client_impl.cc:224] notifyRemoteClose 16 Missing Authentication Token
[2024-06-27 03:40:53.627][17][warning][config] [./source/extensions/config_subscription/grpc/grpc_stream.h:152] StreamAggregatedResources gRPC config stream to appmesh-envoy-management.us-west-2.amazonaws.com:443 closed: 16, Missing Authentication Token
[2024-06-27 03:40:53.627][17][debug][config] [source/extensions/config_subscription/grpc/grpc_subscription_impl.cc:115] gRPC update for type.googleapis.com/envoy.config.cluster.v3.Cluster failed
Looking at the logs in the Pod Intentity I can see this repeated many times:
{"client-addr":"10.0.3.220:59826","cluster-name":"CLUSTER_NAME","level":"info","msg":"handling new request request from 10.0.3.220:59826","time":"2024-06-27T03:43:22Z"}
{"client-addr":"10.0.3.220:59826","cluster-name":"CLUSTER_NAME","level":"error","msg":"Error fetching credentials: Service account token cannot be empty","time":"2024-06-27T03:43:22Z"}
Adding to the point, upstream envoy supported it starting 1.30.0. https://github.com/envoyproxy/envoy/blob/f79b881883e862bc0f7dc7f09d3bc811fb0944f6/changelogs/1.30.0.yaml#L483 Can we have aws-appmesh-envoy image based on 1.30? Thanks
If you want to see App Mesh implement this idea, please upvote with a :+1:.
Tell us about your request I think EKS Pod Identities are not supported at this time for the Envoy containers injected in EKS.
Which integration(s) is this request for? EKS
Tell us about the problem you're trying to solve. What are you trying to do, and why is it hard? I created an app in my EKS cluster and gave permissions using EKS Pod Identities. I'm deploying a Node App with an AppConfig container. It works fine and the permissions are working as expected. However, when I added the AppMesh integration with the Container Injected automatically I receive the following error:
Which causes the AppConfig container to fail trying to fetch the parameters
However, I can see that the env variables in the Envoy container that EKS pod identities inject into containers are correctly set:
This is the whole manifest for this particular container:
I wonder if EKS Pod Identities are not supported at this time or if there is something I can't see.
By the way, the App Role already has permissions for
appmesh:StreamAggregatedResources
with the resource set to the Virtual Node ARN