Document tips around clock skew

[codefromthecrypt]

Q: how do you sync the times on the various services? Do the clients connect to the zipkin server and work out the difference between their clock and the servers? Or are you using some other way of stitching the requests together

A: clock sync is not attacked by Zipkin, there are existing solutions for it (see NTP). But, especially in a virtualized / cloud environment, there will always be clock drift; Zipkin does try to address that when displaying data with various heuristics (like, if this service called that service, then this span must come before that span, and not after, even if the timestamps say differently)

[DanielYWoo]

Even with NTP, it could be 300 ms clock difference between services, but an RPC could only take 30 ms, so the timestamp can only be used to calculate duration, we should never compare timestamps across services. Am I right?

[dineshkapoor27]

Should we have this feature where there is one time which every service follows? Or should the time stamp be one the basis of the entry time of the log in a central log repository (maybe ELK)?

[codefromthecrypt]

Should we have this feature where there is one time which every service follows? Or should the time stamp be one the basis of the entry time of the log in a central log repository (maybe ELK)?

The origin of timestamps are in the apps, so they'd have to have something to control it. I'm not sure the OSS licensable mechanisms out there, but I suspect many many APM vendors have solved this in various ways.

[jcchavezs]

There is definitively clock-skew logic in UI (isn't @tacigar). Maybe we need to add a rationale document around it next to that code? cc @adriancole

[jcchavezs]

Shall this issue be moved to zipkin/zipkin-support? cc @jorgheymans

[jorgheymans]

Sure, that or the new discussions feature ?

[codefromthecrypt]

can do both if you want.. discussions feature in this repo!