I have a AWS EKS Fargate setup and have an ingress Gateway setup that makes call to two micro services. The service are Pemissions and Service-Providers.
X-Ray Service Map does show the calls are made correctly.
So, the call goes from Virtual gateway to permissions and service-providers and is shown on service map.
However, there is another call that happens between these two services themselves that is not routed from the gateway, where service providers contacts permissions to find out whether the user who made the call has permissions to get that data or not depending on his/her auth. token.
This particular call does not shows up in service map. As you can see there is no line connecting the two services. Although in traces I do see that call
The permissions service is referenced as http://ganesh-permissions in the service providers service. What am I missing?
Ingress Gateway X-ray logs
2020-10-12T17:21:51Z [Info] Successfully sent batch of 1 segments (0.004 seconds)
2020-10-12T17:21:52Z [Info] Successfully sent batch of 1 segments (0.004 seconds)
2020-10-12T17:21:54Z [Info] Successfully sent batch of 2 segments (0.013 seconds)
2020-10-12T17:21:57Z [Info] Successfully sent batch of 1 segments (0.007 seconds)
2020-10-12T17:22:04Z [Info] Successfully sent batch of 1 segments (0.006 seconds)
2020-10-12T17:22:06Z [Info] Successfully sent batch of 1 segments (0.014 seconds)
2020-10-12T17:22:07Z [Info] Successfully sent batch of 1 segments (0.016 seconds)
2020-10-12T17:22:09Z [Info] Successfully sent batch of 2 segments (0.013 seconds)
2020-10-12T17:22:12Z [Info] Successfully sent batch of 1 segments (0.014 seconds)
2020-10-12T17:22:19Z [Info] Successfully sent batch of 1 segments (0.004 seconds)
2020-10-12T17:22:21Z [Info] Successfully sent batch of 1 segments (0.004 seconds)
2020-10-12T17:22:22Z [Info] Successfully sent batch of 1 segments (0.013 seconds)
2020-10-12T17:22:24Z [Info] Successfully sent batch of 1 segments (0.004 seconds)
service-providers logs
2020-10-12T16:51:07Z [Info] Initializing AWS X-Ray daemon 3.2.0
2020-10-12T16:51:07Z [Info] Using buffer memory limit of 74 MB
2020-10-12T16:51:07Z [Info] 1184 segment buffers allocated
2020-10-12T16:51:07Z [Info] Using region: us-east-1
2020-10-12T16:52:08Z [Error] Get instance id metadata failed: RequestError: send request failed
caused by: Get http://169.254.169.254/latest/meta-data/instance-id: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)
2020-10-12T16:52:08Z [Info] HTTP Proxy server using X-Ray Endpoint : https://xray.us-east-1.amazonaws.com
2020-10-12T16:52:08Z [Info] Starting proxy http server on 0.0.0.0:2000
2020-10-12T16:52:10Z [Info] Successfully sent batch of 1 segments (0.285 seconds)
2020-10-12T16:52:10Z [Info] Successfully sent batch of 1 segments (0.014 seconds)
2020-10-12T17:05:53Z [Info] Successfully sent batch of 1 segments (0.012 seconds)
2020-10-12T17:14:02Z [Info] Successfully sent batch of 1 segments (0.016 seconds)
permissions logs
[2020-10-12 01:53:18.664][1][info][upstream] [source/common/upstream/cds_api_impl.cc:65] cds: add 3 cluster(s), remove 0 cluster(s)
[2020-10-12 01:53:18.667][1][warning][misc] [source/common/protobuf/message_validator_impl.cc:21] Deprecated field: type envoy.api.v2.listener.Filter Using deprecated option 'envoy.api.v2.listener.Filter.config' from file listener_components.proto. This configuration will be removed from Envoy soon. Please see https://www.envoyproxy.io/docs/envoy/latest/version_history/version_history for details. If continued use of this field is absolutely necessary, see https://www.envoyproxy.io/docs/envoy/latest/configuration/operations/runtime#using-runtime-overrides-for-deprecated-features for how to apply a temporary and highly discouraged override.
[2020-10-12 01:53:18.668][1][warning][misc] [source/common/protobuf/message_validator_impl.cc:21] Deprecated field: type envoy.api.v2.listener.Filter Using deprecated option 'envoy.api.v2.listener.Filter.config' from file listener_components.proto. This configuration will be removed from Envoy soon. Please see https://www.envoyproxy.io/docs/envoy/latest/version_history/version_history for details. If continued use of this field is absolutely necessary, see https://www.envoyproxy.io/docs/envoy/latest/configuration/operations/runtime#using-runtime-overrides-for-deprecated-features for how to apply a temporary and highly discouraged override.
[2020-10-12 01:53:18.668][1][warning][misc] [source/common/protobuf/message_validator_impl.cc:21] Deprecated field: type envoy.api.v2.listener.Filter Using deprecated option 'envoy.api.v2.listener.Filter.config' from file listener_components.proto. This configuration will be removed from Envoy soon. Please see https://www.envoyproxy.io/docs/envoy/latest/version_history/version_history for details. If continued use of this field is absolutely necessary, see https://www.envoyproxy.io/docs/envoy/latest/configuration/operations/runtime#using-runtime-overrides-for-deprecated-features for how to apply a temporary and highly discouraged override.
[2020-10-12 01:53:18.668][1][warning][misc] [source/common/protobuf/message_validator_impl.cc:21] Deprecated field: type envoy.api.v2.listener.Filter Using deprecated option 'envoy.api.v2.listener.Filter.config' from file listener_components.proto. This configuration will be removed from Envoy soon. Please see https://www.envoyproxy.io/docs/envoy/latest/version_history/version_history for details. If continued use of this field is absolutely necessary, see https://www.envoyproxy.io/docs/envoy/latest/configuration/operations/runtime#using-runtime-overrides-for-deprecated-features for how to apply a temporary and highly discouraged override.
[2020-10-12 02:27:53.661][1][warning][config] [bazel-out/k8-opt/bin/source/common/config/_virtual_includes/grpc_stream_lib/common/config/grpc_stream.h:93] StreamAggregatedResources gRPC config stream closed: 0,
[2020-10-12 03:02:48.122][1][warning][config] [bazel-out/k8-opt/bin/source/common/config/_virtual_includes/grpc_stream_lib/common/config/grpc_stream.h:93] StreamAggregatedResources gRPC config stream closed: 0,
[2020-10-12 03:36:04.981][1][warning][config] [bazel-out/k8-opt/bin/source/common/config/_virtual_includes/grpc_stream_lib/common/config/grpc_stream.h:93] StreamAggregatedResources gRPC config stream closed: 0,
[2020-10-12 04:07:31.442][1][warning][config] [bazel-out/k8-opt/bin/source/common/config/_virtual_includes/grpc_stream_lib/common/config/grpc_stream.h:93] StreamAggregatedResources gRPC config stream closed: 0,
[2020-10-12 04:39:01.991][1][warning][config] [bazel-out/k8-opt/bin/source/common/config/_virtual_includes/grpc_stream_lib/common/config/grpc_stream.h:93] StreamAggregatedResources gRPC config stream closed: 0,
[2020-10-12 05:13:44.431][1][warning][config] [bazel-out/k8-opt/bin/source/common/config/_virtual_includes/grpc_stream_lib/common/config/grpc_stream.h:93] StreamAggregatedResources gRPC config stream closed: 0,
[2020-10-12 05:48:14.121][1][warning][config] [bazel-out/k8-opt/bin/source/common/config/_virtual_includes/grpc_stream_lib/common/config/grpc_stream.h:93] StreamAggregatedResources gRPC config stream closed: 0,
[2020-10-12 06:19:34.790][1][warning][config] [bazel-out/k8-opt/bin/source/common/config/_virtual_includes/grpc_stream_lib/common/config/grpc_stream.h:93] StreamAggregatedResources gRPC config stream closed: 0,
[2020-10-12 06:50:44.821][1][warning][config] [bazel-out/k8-opt/bin/source/common/config/_virtual_includes/grpc_stream_lib/common/config/grpc_stream.h:93] StreamAggregatedResources gRPC config stream closed: 0,
[2020-10-12 07:22:03.401][1][warning][config] [bazel-out/k8-opt/bin/source/common/config/_virtual_includes/grpc_stream_lib/common/config/grpc_stream.h:93] StreamAggregatedResources gRPC config stream closed: 0,
[2020-10-12 07:55:42.221][1][warning][config] [bazel-out/k8-opt/bin/source/common/config/_virtual_includes/grpc_stream_lib/common/config/grpc_stream.h:93] StreamAggregatedResources gRPC config stream closed: 0,
[2020-10-12 08:30:58.151][1][warning][config] [bazel-out/k8-opt/bin/source/common/config/_virtual_includes/grpc_stream_lib/common/config/grpc_stream.h:93] StreamAggregatedResources gRPC config stream closed: 0,
[2020-10-12 09:05:50.581][1][warning][config] [bazel-out/k8-opt/bin/source/common/config/_virtual_includes/grpc_stream_lib/common/config/grpc_stream.h:93] StreamAggregatedResources gRPC config stream closed: 0,
[2020-10-12 09:37:48.561][1][warning][config] [bazel-out/k8-opt/bin/source/common/config/_virtual_includes/grpc_stream_lib/common/config/grpc_stream.h:93] StreamAggregatedResources gRPC config stream closed: 0,
[2020-10-12 10:11:15.850][1][warning][config] [bazel-out/k8-opt/bin/source/common/config/_virtual_includes/grpc_stream_lib/common/config/grpc_stream.h:93] StreamAggregatedResources gRPC config stream closed: 0,
[2020-10-12 10:46:04.822][1][warning][config] [bazel-out/k8-opt/bin/source/common/config/_virtual_includes/grpc_stream_lib/common/config/grpc_stream.h:93] StreamAggregatedResources gRPC config stream closed: 0,
[2020-10-12 11:18:06.742][1][warning][config] [bazel-out/k8-opt/bin/source/common/config/_virtual_includes/grpc_stream_lib/common/config/grpc_stream.h:93] StreamAggregatedResources gRPC config stream closed: 0,
[2020-10-12 11:52:18.991][1][warning][config] [bazel-out/k8-opt/bin/source/common/config/_virtual_includes/grpc_stream_lib/common/config/grpc_stream.h:93] StreamAggregatedResources gRPC config stream closed: 0,
[2020-10-12 12:23:54.331][1][warning][config] [bazel-out/k8-opt/bin/source/common/config/_virtual_includes/grpc_stream_lib/common/config/grpc_stream.h:93] StreamAggregatedResources gRPC config stream closed: 0,
[2020-10-12 12:55:58.631][1][warning][config] [bazel-out/k8-opt/bin/source/common/config/_virtual_includes/grpc_stream_lib/common/config/grpc_stream.h:93] StreamAggregatedResources gRPC config stream closed: 0,
[2020-10-12 13:26:12.482][1][warning][config] [bazel-out/k8-opt/bin/source/common/config/_virtual_includes/grpc_stream_lib/common/config/grpc_stream.h:93] StreamAggregatedResources gRPC config stream closed: 0,
[2020-10-12 13:58:35.981][1][warning][config] [bazel-out/k8-opt/bin/source/common/config/_virtual_includes/grpc_stream_lib/common/config/grpc_stream.h:93] StreamAggregatedResources gRPC config stream closed: 0,
[2020-10-12 14:30:32.361][1][warning][config] [bazel-out/k8-opt/bin/source/common/config/_virtual_includes/grpc_stream_lib/common/config/grpc_stream.h:93] StreamAggregatedResources gRPC config stream closed: 0,
[2020-10-12 15:03:47.371][1][warning][config] [bazel-out/k8-opt/bin/source/common/config/_virtual_includes/grpc_stream_lib/common/config/grpc_stream.h:93] StreamAggregatedResources gRPC config stream closed: 0,
[2020-10-12 15:38:25.851][1][warning][config] [bazel-out/k8-opt/bin/source/common/config/_virtual_includes/grpc_stream_lib/common/config/grpc_stream.h:93] StreamAggregatedResources gRPC config stream closed: 0,
[2020-10-12 16:08:48.791][1][warning][config] [bazel-out/k8-opt/bin/source/common/config/_virtual_includes/grpc_stream_lib/common/config/grpc_stream.h:93] StreamAggregatedResources gRPC config stream closed: 0,
[2020-10-12 16:43:54.471][1][warning][config] [bazel-out/k8-opt/bin/source/common/config/_virtual_includes/grpc_stream_lib/common/config/grpc_stream.h:93] StreamAggregatedResources gRPC config stream closed: 0,
[2020-10-12 17:16:26.841][1][warning][config] [bazel-out/k8-opt/bin/source/common/config/_virtual_includes/grpc_stream_lib/common/config/grpc_stream.h:93] StreamAggregatedResources gRPC config stream closed: 0,
I have a AWS EKS Fargate setup and have an ingress Gateway setup that makes call to two micro services. The service are Pemissions and Service-Providers.
X-Ray Service Map does show the calls are made correctly.
So, the call goes from Virtual gateway to permissions and service-providers and is shown on service map.
However, there is another call that happens between these two services themselves that is not routed from the gateway, where service providers contacts permissions to find out whether the user who made the call has permissions to get that data or not depending on his/her auth. token.
This particular call does not shows up in service map. As you can see there is no line connecting the two services. Although in traces I do see that call
The permissions service is referenced as http://ganesh-permissions in the service providers service. What am I missing?
Ingress Gateway X-ray logs
service-providers logs
permissions logs