Is it possible to use Loguru with OpenTelemetry Logging Instrumentation?

[luisazevedo-mb]

I'm trying to configure Loguru with OpenTelemetry Logging Instrumentation, but I'm not getting. I can see that LoggingInstrumentor().instrument() get the default factory of loggin and than add some controls, changing the default factory of logging. I'm not able to find a way to define the factory for Loguru.

Block the code executed by LoggingInstrumentor().instrument()

    def _instrument(self, **kwargs):

        provider = kwargs.get("tracer_provider", None) or get_tracer_provider()
        old_factory = logging.getLogRecordFactory()
        LoggingInstrumentor._old_factory = old_factory

        service_name = None

        def record_factory(*args, **kwargs):
            record = old_factory(*args, **kwargs)

            record.otelSpanID = "0"
            record.otelTraceID = "0"

            nonlocal service_name
            if service_name is None:
                resource = getattr(provider, "resource", None)
                if resource:
                    service_name = (
                        resource.attributes.get("service.name") or ""
                    )
                else:
                    service_name = ""

            record.otelServiceName = service_name

            span = get_current_span()
            if span != INVALID_SPAN:
                ctx = span.get_span_context()
                if ctx != INVALID_SPAN_CONTEXT:
                    record.otelSpanID = format(ctx.span_id, "016x")
                    record.otelTraceID = format(ctx.trace_id, "032x")
            return record

        logging.setLogRecordFactory(record_factory)

        set_logging_format = kwargs.get(
            "set_logging_format",
            environ.get(OTEL_PYTHON_LOG_CORRELATION, "false").lower()
            == "true",
        )

        if set_logging_format:
            log_format = kwargs.get(
                "logging_format", environ.get(OTEL_PYTHON_LOG_FORMAT, None)
            )
            log_format = log_format or DEFAULT_LOGGING_FORMAT

            log_level = kwargs.get(
                "log_level", LEVELS.get(environ.get(OTEL_PYTHON_LOG_LEVEL))
            )
            log_level = log_level or logging.INFO

            logging.basicConfig(format=log_format, level=log_level)

[Delgan]

Hi @luisazevedo-mb.

I'm not sure it work but what about using InterceptHandler or PropagateHandler (depending on your goal) to integrate Loguru with standard logging module?

Otherwise you may need to re-implement the record_factory() in the snippet you shared in way compatible with Loguru. It's just about retrieving the contextual information and adding them to the extra dict.

[phitoduck]

+1 for this.

I was working with NewRelic for work the other day and discovered that NewRelic's auto-instrumentation CLI does support loguru.

Here is the PR in the newrelic/newrelic-python-agent repo where loguru instrumentation was added: https://github.com/newrelic/newrelic-python-agent/pull/552

I think this could be a good reference resource.

[Corfucinas]

This would be a great thing to have on the next release

[elanqo]

@luisazevedo-mb were you able to integrate Loguru and Open Telemetry? I'm facing a similar issue. I want logs in Loguru format propagate all the way to open telemetry.

[axiangcoding]

I also need this feature, any plan for this?

[Delgan]

Apologies to everyone eagerly anticipating this feature, but not being familiar with OpenTelemetry I'm still not sure about what exactly is expected. If someone could please clarify the desired output of the integration between Loguru and OpenTelemetry, perhaps I could craft a working solution.

If I understand correctly, the LoggingInstrumentor() from opentelemetry can be used this way:

import logging

from opentelemetry.instrumentation.logging import LoggingInstrumentor

LoggingInstrumentor().instrument(set_logging_format=True)

logger = logging.getLogger(__name__)

logger.info("This is an info message")

This produces the following logs:

2024-01-02 18:31:27,332 INFO [__main__] [file.py:9] [trace_id=0 span_id=0 resource.service.name= trace_sampled=False] - This is an info message

We can observe some contextual data was automatically added. Is that what you try to propagate to Loguru? If this is the case, we can easily re-implement an "instrumentor" through the patch() method of Loguru. Here is an example based on the original LoggingInstrumentor._instrument() implementation:

from loguru import logger
import sys

from opentelemetry.trace import (
    INVALID_SPAN,
    INVALID_SPAN_CONTEXT,
    get_current_span,
    get_tracer_provider,
)

def instrument_loguru():
    provider = get_tracer_provider()
    service_name = None

    def add_trace_context(record):
        record["extra"]["otelSpanID"] = "0"
        record["extra"]["otelTraceID"] = "0"
        record["extra"]["otelTraceSampled"] = False

        nonlocal service_name
        if service_name is None:
            resource = getattr(provider, "resource", None)
            if resource:
                service_name = resource.attributes.get("service.name") or ""
            else:
                service_name = ""

        record["extra"]["otelServiceName"] = service_name

        span = get_current_span()
        if span != INVALID_SPAN:
            ctx = span.get_span_context()
            if ctx != INVALID_SPAN_CONTEXT:
                record["extra"]["otelSpanID"] = format(ctx.span_id, "016x")
                record["extra"]["otelTraceID"] = format(ctx.trace_id, "032x")
                record["extra"]["otelTraceSampled"] = ctx.trace_flags.sampled

    logger.configure(patcher=add_trace_context)

instrument_loguru()

logger.remove()
format_ = "{time:YYYY-MM-DD HH:MM:SS.sss} {level} [{name}] [{file}:{line} [trace_id={extra[otelTraceID]} span_id={extra[otelSpanID]} resource.service.name={extra[otelServiceName]} trace_sampled={extra[otelTraceSampled]}] - {message}"
logger.add(sys.stderr, format=format_)

logger.info("This is an info message")

This would produce the same output as with the standard logging library.

[s71m]

Inspired by: https://github.com/Delgan/loguru/issues/674#issuecomment-1874389387

Opentelemetry loguru handler: https://github.com/s71m/opentelemetry-loguru-telegram/blob/master/utils/loguru_otlp_handler.py