Open ddluke opened 5 months ago
@ddluke
This looks very promising. Apologies for the late response.
is a LogRecordProcessor the right weapon of choice to accomplish this? (as of now, I do not see that logging.Formatters are already supported, and honestly, using a LogRecordProcessor also feels a lot cleaner then messing with the untyped interfaces of formatters π π¬ )
This certainly looks like an acceptable way to achieve this behavior. Another way is to natively support logs that are built on pydantic in the Otel LoggingHandler itself. This can either be added to the already existing LoggingHandler
or we can create a new handler every time we want to support other logging formats. The advantage with your proposal is that it is much more modularized and the LogRecordProcessor can be shipped as a separate component. This can also be seen as a "disadvantage" if users would not want to add any more components to their code and just want structured logs to be supported out of the box.
is the return type annotation for above processor defined correctly? (the base LogRecordProcessor does not have a return type annotation at all in it's emit method)
We are working on adding typing to our sdk. The emit
method does not return anything. See this as well for a simple example. Processors are usually used to perform some sort of pre/post operation on the telemetry data.
are the shutdown and force_flush methods setup correctly for a log processors whose only purpose is to do some modifications to the log data and emit it to the pipeline?
Yes this looks right to me.
Related feature request thread on adding structlog
handler.
Hey folks :)
I've been eyeballing this project and the otel specification for months, since sooner or later we want to replace our own telemetry sdks (build on tools like structlog and pydantic) in our distributed cloud MLOps platform with otel. I feel this is the right thing to do.
I know the logs sdk is still not stable, but I have started to do hands on experiments nonetheless, by recreating the otel provided docker compose stack with my own apps to get a feeling on how logs, metrics and traces are defined and utilised by otel. This helps me to already layout the roadmap for migrating to otel and prepare all user stories for this epic.
One thing that we do not want to change when moving to otel, is the way how our structured logs are defined, as that has proven of great value across all components of our platform (restful services, lambda functions, containerised apps, batch processing and training jobs, scheduler environments, realtime inference endpoints, etc.).
We currently model our log messages (or rather the log bodies) using pydantic, and emit them by passing instantiated pydantic models to our logging sdk, somewhat similar to this:
This way of logging is established across all platform components and has been adopted by all people involved (Data Scientists, Data Engineers, etc.). It enables a very high degree of consistency and allows us to do advanced operations, like sending all of our logs to a data catalogue and query it via sql (to mention one use case).
After some hours of reading the docs, reverse engineering the logs sdk and iterating over my own docker compose stack, I feel like the correct approach to add support for pydantic based log messages, is using a
LogRecordProcessor
.The following snipped roughly demonstrates what we might want to do:
If I spin up a local otel collector (similar to how it is configured by the logs sample in this repo), I do see an output like I would expect it:
This is great, but I do have some open questions:
LogRecordProcessor
the right weapon of choice to accomplish this? (as of now, I do not see thatlogging.Formatters
are already supported, and honestly, using aLogRecordProcessor
also feels a lot cleaner then messing with the untyped interfaces of formatters π π¬ )LogRecordProcessor
does not have a return type annotation at all in it's emit method)shutdown
andforce_flush
methods setup correctly for a log processors whose only purpose is to do some modifications to the log data and emit it to the pipeline?Cheers and kind regards π