search

aws / aws-xray-daemon

The AWS X-Ray daemon listens for traffic on UDP port 2000, gathers raw segment data, and relays it to the AWS X-Ray API.
Apache License 2.0
190 stars 69 forks source link

Lag when adding aws xray daemon #183

Open jpresley23-tw opened 1 year ago

jpresley23-tw commented 1 year ago

We saw an increase of about 3 seconds when we started to send transactions to the aws xray daemon from our spring boot application. While debugging with time nc -vz -u localhost 2000 we see there is a 3s duration for the connection when we add the aws xray daemon as a sidecar to our debug pod. We get the same result when we try to run the aws xray daemon as a deployment. We tried to change the port but got the same result. Is there a way to try to use TCP protocol instead of UDP protocol for the aws xray daemon or is there something else we can try to remove the lag we noticed after we added the aws xray daemon?

wangzlei commented 1 year ago

So, we are not sure the lag would be solved by TCP or Daemon improvement. Could you provide the info of time nc -vz -u localhost 2000 and how do you instrument spring boot?

jpresley23-tw commented 1 year ago

Since posting my issue I went back to using a deployment with a service in front of it for the X-Ray Daemon and I tried changing the port to 3000. I get the same results. The output is

time nc -vz -u aws-xray-service.aws-xray 3000
Connection to aws-xray-service.aws-xray (10.100.47.15) 3000 port [udp/*] succeeded!

real    0m3.016s
user    0m0.002s
sys 0m0.000s

We instrumented spring boot with: in application.yml

  zipkin:
    internal:
      module:
        xray: zipkin.module.aws.xray.ZipkinXRayStorageModule
    storage:
      xray:
        daemon-address: aws-xray-service.aws-xray:2000

and in build.gradle

implementation 'io.zipkin.aws:zipkin-reporter-xray-udp:0.23.4'