[Question] Loguru handler with Opentelemetry: Invalid type dict for attribute 'extra' value. Expected one of ['bool', 'str', 'bytes', 'int', 'float'] or a sequence of those types (hashtag1063)

[vikramsubramanian]

When using opentelemetry-collter to collect logs, loguru will throw a warning like this: Invalid type dict for attribute 'extra' value. Expected one of ['bool', 'str', 'bytes', 'int' , 'float'] or a sequence of those types

hashtaghashtag my code:

import logging

from loguru import logger
from opentelemetry import trace
from opentelemetry._logs import set_logger_provider
from opentelemetry.exporter.otlp.proto.grpc._log_exporter import (
    OTLPLogExporter,
)
from opentelemetry.sdk._logs import LoggerProvider, LoggingHandler
from opentelemetry.sdk._logs.export import BatchLogRecordProcessor
from opentelemetry.sdk.resources import Resource
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk.trace.export import (
    BatchSpanProcessor,
    ConsoleSpanExporter,
)

if __name__ == '__main__':
    trace.set_tracer_provider(TracerProvider())
    trace.get_tracer_provider().add_span_processor(
        BatchSpanProcessor(ConsoleSpanExporter())
    )

    logger_provider = LoggerProvider(
        resource=Resource.create(
            {
                "service.name": "shoppingcart",
                "service.instance.id": "instance-12",
            }
        ),
    )
    set_logger_provider(logger_provider)

    exporter = OTLPLogExporter(insecure=True)
    logger_provider.add_log_record_processor(BatchLogRecordProcessor(exporter))
    handler = LoggingHandler(level=logging.NOTSET, logger_provider=logger_provider)

    hashtag remove console
    logger.remove()
    logger.add(handler, serialize=True)
    logger.info('logs without trace')

    hashtag Trace context correlation
    tracer = trace.get_tracer(__name__)
    with tracer.start_as_current_span("foo"):
        hashtag Do something
        hashtag logger2.error("Hyderabad, we have a major problem.")
        logger.error("Hyderabad, we have a major problem.")

    logger_provider.shutdown()

hashtaghashtag console output: ![截屏2024-01-05 17 33 40](

hashtaghashtag env:

hashtag pyproject.toml

[tool.poetry.dependencies]
python = "^3.12"
opentelemetry-api = "^1.22.0"
opentelemetry-sdk = "^1.22.0"
opentelemetry-exporter-jaeger = "^1.21.0"
pymongo = "^4.6.1"
opentelemetry-instrumentation = "^0.43b0"
opentelemetry-distro = "^0.43b0"
opentelemetry-exporter-otlp-proto-grpc = "^1.22.0"
opentelemetry-instrumentation-pymongo = "^0.43b0"
opentelemetry-exporter-otlp = "^1.22.0"
loguru = "^0.7.2"
opentelemetry-instrumentation-logging = "^0.43b0"

system: macOS 14.2.1 (23C71)

)

[mayil-ai[bot]]

The warning Invalid type dict for attribute 'extra' value. Expected one of ['bool', 'str', 'bytes', 'int', 'float'] or a sequence of those types suggests that the extra attribute of a log record, which is being passed to OpenTelemetry, is expected to be a simple type or a sequence of simple types, but a dictionary is being provided instead.

To resolve this issue, you should ensure that the extra attribute of the log records being sent to OpenTelemetry contains only the expected types. If you need to include a dictionary, you should serialize it into a string or flatten it into multiple key-value pairs that conform to the expected types.

Here is a high-level overview of how you might modify the extra attribute before sending it to OpenTelemetry:

• Serialize the dictionary into a JSON string.
• Flatten the dictionary into individual key-value pairs with keys as strings and values as simple types.

Here is a pseudo-code example of how you might serialize the extra dictionary:

import json

# Assuming log_record is the record being processed
if isinstance(log_record['extra'], dict):
    log_record['extra'] = json.dumps(log_record['extra'])

And here is an example of how you might flatten the dictionary:

# Assuming log_record is the record being processed
if isinstance(log_record['extra'], dict):
    for key, value in log_record['extra'].items():
        if isinstance(value, (bool, str, bytes, int, float)):
            log_record[f'extra_{key}'] = value
    del log_record['extra']  # Remove the original 'extra' key

You would need to integrate this logic into the appropriate place in your logging pipeline, ensuring that the log records are modified before they are sent to OpenTelemetry.