search

aws / aws-xray-daemon

The AWS X-Ray daemon listens for traffic on UDP port 2000, gathers raw segment data, and relays it to the AWS X-Ray API.
Apache License 2.0
189 stars 69 forks source link

XRAY-DAEMON Behind Load Balancer #94

Open somnathg-fullstack opened 3 years ago

somnathg-fullstack commented 3 years ago

How to put XRAY-DAEMON behind a load balancer?

willarmiros commented 3 years ago

@somnathg-fullstack we will need more context than that to provide meaningful guidance. There are instructions on how to run the X-Ray daemon on EC2, ECS, and Elastic Beanstalk here: https://docs.aws.amazon.com/xray/latest/devguide/xray-daemon.html

I would suggest consulting the documentation relevant to you depending on what infrastructure is sitting behind your LB, then if you have trouble with implementation, please post a detailed explanation of your setup and the exact issue you are experiencing.

trav-c commented 3 years ago

@willarmiros

I'm curious about this too, right now in our EC2 ECS cluster we run the xray-daemon as an ECS daemon task, listening on 0.0.0.0:2000 then point all our other ECS containers at the host end of the docker bridge (172.17.0.1:2000) so they can share one xray daemon per EC2 container instance.

However now that ALBs support UDP listeners, I'd be interested to know if it's viable to run the xray-daemon as a replica service and use a UDP listener on an ALB to expose the service group to all the tasks in the cluster so as to reduce the memory overhead by not running an xray-daemon on every EC2 Container Instance, just running however many are needed based on something like a CPU target-tracking autoscaling policy.

It's particularly relevant to us now because we're hitting frequent OOMs for the xray-daemon container with an allocation of 256M, so are looking to potentially increasing that, but considering trying to offset the additional overhead by running fewer copies of the daemon.

So I guess my specific question, which may or may not be @somnathg-fullstack's scenario is would the following deployment model be expected to work and be fully functional:

[Microservice ECS Task] ==> [ALB:UDP Listener] ==> [ECS Replica Service running xray-daemon]

Where both the the Microservice tasks and the ECS replica service tasks would be on the same EC2 hosted ECS cluster in our case.

stale[bot] commented 2 years ago

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs in next 7 days. Thank you for your contributions.

dev-for-uniq commented 2 years ago

@willarmiros, does @trav-c question have a valid answer as I am also thinking on similar terms ??