Distributed - event driven architecture

[berlinger-ggeorgitsis]

Is it possible to use this library for distributed systems? In a scenario where X independent services are collaborating with each other via HTTP calls and/or events via a broker, can the TraceId be passed along and be used to alter/add segments to a started Trace?

[patrickkerrigan]

Hi George,

Yep, it's definitely possible via HTTP calls, and works roughly as you think it would. I haven't personally tried using events via a broker, but I don't see a reason why that wouldn't work.

I'd recommend taking a look at https://github.com/patrickkerrigan/php-xray/issues/11#issuecomment-953736831 which deals with how to make it work over HTTP calls to other services.

The main steps are:

• Create a HttpSegment (or RemoteSegment) in your main request
• Call ->setTraced(true) to indicate that x-ray should wait for a child trace
• Pass the x-ray header through to your remote service, adding the Parent key with a value of the ID of the segment you created above
• In your remote service, initialise x-ray with the header you passed through
• Back in the main request, wait for a response from your remote service then close the segment

X-ray should then link all of the traces together and you'll be able to view them all as a single trace, with the different contributing services marked out.

The one thing I'm not sure of is if x-ray will allow a child trace to end after the main trace, e.g if you kick off a background job via a broker that completes after the main request.

[berlinger-ggeorgitsis]

Hi Patrick,

thanks a lot for your message.

I tried some hands-on functionality with your implementation and it works slightly different and luckily that's for a good reason. Indeed the communication link between the services is the X-Amzn-Trace-Id either injected in the HTTP header or transferred via any other broker way. The value of the TraceID is what matters actually.

Every service must start and end a trace, without waiting for the remote one to close the segment (unless it's a sub-segment in the same script). The only thing that matters is that the services(which run in their own processes, they are on a different server) should share the same TraceID between them. Then based on the timings (start-end) of every service that started-ended a trace, AWS will link them and display them properly. It's very important that every service needs to end the trace it started, otherwise they are not visible on X-Ray.

So in practise there are no child traces, every trace is a stand alone one and if you manage to submit them with a TraceID that already exists, X-Ray will take care of the rest.

Thanks again for your time.

[patrickkerrigan]

Thanks for getting back to me with your findings!

Interesting that it's just the trace ID that matters. I'll have to give this a go at some point, would definitely make it easier to trace asynchronous tasks that don't finish during the web request.