search

itamarst / eliot

Eliot: the logging system that tells you *why* it happened
https://eliot.readthedocs.io
Apache License 2.0
1.1k stars 66 forks source link

Ideas from OpenTracing and OpenCensus #361

Open ghost opened 5 years ago

ghost commented 5 years ago

OpenTracing (opentracing-python and opentracing-python-instrumentation) and OpenCensus (opencensus-python) seem to have some similar ideas about structured tracing/logging, and apparently aim to provide a standardized way to record information across multiple languages.

I'm not really familiar with these tools -- perhaps you already are. I wondered if you might be interested in cribbing some ideas from them or comparing their offerings to Eliot's.

itamarst commented 5 years ago

I'm less familiar with OpenCensus, but it seems similar to OpenTracing. In both cases the goal is to address the needs of large scale distributed systems in very large organizations. OpenCensus came out of Google looks like, OpenTracing is a consensus design with a bunch of large scale vendors.

Difference 1: Presumed scale

Both OpenTracing and OpenCensus come out of large organizations.

In theory they both scale down, but they're really designed for larger organizations using multiple languages. Like, for OpenTracing I'm not even sure how I would just do the equivalent of https://eliot.readthedocs.io/en/stable/quickstart.html locally with OpenTracing.

I assume it's possible, but first you have to wade through the difference between abstract provider and implementation-specific stuff, and the fact everyone involved wants to sell you their SaaS log hosting, or at least assume you have someone devoted full time to running infrastructure like Zipkin.

https://opencensus.io/quickstart/python/tracing/ is somewhat better... but step 1 is still "Now setup Zipkin".

Difference 2: Language agnostic

There's a bunch of very Python-specific stuff in Eliot. Like, it knows how to extract info from exceptions, so e.g. OSError it'll extract the errno and log it as a field on its own. OpenTracing and OpenCensus try to be generic, which has pluses (works elsewhere) and minuses. It's possible to port Eliot to other languages (someone did first pass at JS), but I want to have a nice Python experience as a first pass.

Random thoughts

Might be nice to utilize all the OpenTracing Python libraries to hook info into Eliot...

itamarst commented 5 years ago

Another use case is feeding data from Eliot to OpenTracing.

jtrakk commented 5 years ago

FYI "Merging OpenTracing and OpenCensus: A Roadmap to Convergence"