search

stripe / veneur

A distributed, fault-tolerant pipeline for observability data
MIT License
1.73k stars 174 forks source link

Readme is not clear whether Is Opentracing a first class citizen in venur #691

Open etsangsplk opened 5 years ago

etsangsplk commented 5 years ago

Documentation on website and readme in repo mostly talk about metrics and only sacntly mention trace. But notice lighstep also one of its backend. Can you put more content and street that how tracing is being treated by the project?

ChimeraCoder commented 5 years ago

Tracing data is definitely a first-class citizen of Veneur. The SSF, which is the native data format that Veneur speaks (although not the only one it can understand), is structured around the concept of a trace span, and that trace span may additionally have metrics attached to it. (More generally: Veneur provides a unified observability pipeline, so metrics and traces are treated as different aspects of the same data.)

Veneur is capable of emitting this data to multiple different backends, which include third-party vendors and open-source tools). Most storage backends that are available are designed to index either metrics or traces, and Veneur routes the data accordingly. In other words, given an SSFSpan which represents a trace span and also has metrics attached in the same protobuf packet, Veneur will send the span data to each configured tracing sink (e.g. Lightstep, AWS X-Ray, Splunk) and the metric data to each configured metric sink (e.g. Datadog, SignalFx).

We recommend using the SSF as a means to emit data (both metrics and traces) to Veneur, as it's the simplest and easiest way to take advantage of everything it has to offer. That said, Veneur also is capable of receiving data along other formats as well.