open-telemetry / opentelemetry-python

OpenTelemetry Python API and SDK
https://opentelemetry.io
Apache License 2.0
1.72k stars 606 forks source link

Logs: SchemaURL is missing in the Resource when using OTLP exporter #3553

Closed tigrannajaryan closed 6 months ago

tigrannajaryan commented 9 months ago

Describe your environment Mac OS X Python 3.9.6

Steps to reproduce

  1. Run Collector, with OTLP gRPC receiver and debug output.
  2. Run example:
    
    import logging

from opentelemetry import trace from opentelemetry import _logs

from opentelemetry._logs import set_logger_provider from opentelemetry.exporter.otlp.proto.grpc._log_exporter import ( OTLPLogExporter, ) from opentelemetry.sdk._logs import LoggerProvider, LoggingHandler from opentelemetry.sdk._logs.export import BatchLogRecordProcessor from opentelemetry.sdk.resources import Resource from opentelemetry.sdk.trace import TracerProvider from opentelemetry.sdk.trace.export import ( BatchSpanProcessor, ConsoleSpanExporter, )

trace.set_tracer_provider(TracerProvider()) trace.get_tracer_provider().add_span_processor( BatchSpanProcessor(ConsoleSpanExporter()) )

logger_provider = LoggerProvider( resource=Resource.create( { "service.name": "shoppingcart2", "service.instance.id": "instance-23", },

Set SchemaURL

    "http://opentelemetry.io/schemas/1.0.0"
),

) set_logger_provider(logger_provider)

exporter = OTLPLogExporter(insecure=True) logger_provider.add_log_record_processor(BatchLogRecordProcessor(exporter)) handler = LoggingHandler(level=logging.NOTSET, logger_provider=logger_provider)

Attach OTLP handler to root logger

logging.getLogger().addHandler(handler)

Log directly

logging.info("Jackdaws love my big sphinx of quartz.")

Create different namespaced loggers

logger1 = logging.getLogger("myapp.area1") logger2 = logging.getLogger("myapp.area2")

logger1.debug("Quick zephyrs blow, vexing daft Jim.") logger1.info("How quickly daft jumping zebras vex.") logger2.warning("Jail zesty vixen who grabbed pay from quack.") logger2.error("The five boxing wizards jump quickly.")

Trace context correlation

tracer = trace.get_tracer(name) with tracer.start_as_current_span("foo"):

Do something

logger2.error("Hyderabad, we have a major problem.")

logger_provider.shutdown()


**What is the expected behavior?**
Resource's SchemaURL should be present.

**What is the actual behavior?**
Resource's SchemaURL is missing:

2023-11-22T13:19:16.798-0500 info ResourceLog #0 Resource SchemaURL: Resource attributes: -> telemetry.sdk.language: Str(python) -> telemetry.sdk.name: Str(opentelemetry) -> telemetry.sdk.version: Str(1.21.0) -> service.name: Str(shoppingcart2) -> service.instance.id: Str(instance-23) ScopeLogs #0 ScopeLogs SchemaURL: InstrumentationScope opentelemetry.sdk._logs._internal LogRecord #0 ObservedTimestamp: 1970-01-01 00:00:00 +0000 UTC Timestamp: 2023-11-22 18:19:16.742461952 +0000 UTC SeverityText: WARNING SeverityNumber: Warn(13) Body: Str(Jail zesty vixen who grabbed pay from quack.) Trace ID: Span ID: Flags: 0



**Additional context**
Part of review https://github.com/open-telemetry/community/issues/1751
hirotasoshu commented 7 months ago

@tigrannajaryan @lzchen Looks like it's missing not only in logs, but also in traces and maybe in metrics (didn't check it) modified example:

import logging

from opentelemetry import trace
from opentelemetry._logs import set_logger_provider
from opentelemetry.exporter.otlp.proto.grpc._log_exporter import (
    OTLPLogExporter,
)
from opentelemetry.exporter.otlp.proto.grpc.trace_exporter import (
    OTLPSpanExporter,
)
from opentelemetry.sdk._logs import LoggerProvider, LoggingHandler
from opentelemetry.sdk._logs.export import (
    BatchLogRecordProcessor,
    ConsoleLogExporter,
)
from opentelemetry.sdk.resources import Resource
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk.trace.export import (
    BatchSpanProcessor,
    ConsoleSpanExporter,
)

resource = Resource.create(
    {
        "service.name": "shoppingcart",
        "service.instance.id": "instance-12",
    },
    "http://opentelemetry.io/schemas/1.0.0",
)

trace.set_tracer_provider(
    TracerProvider(
        resource=resource
    )
)
trace.get_tracer_provider().add_span_processor(
    BatchSpanProcessor(OTLPSpanExporter())
)

logger_provider = LoggerProvider(
    resource=resource
)
set_logger_provider(logger_provider)

exporter = OTLPLogExporter(insecure=True)
# exporter = ConsoleLogExporter()
logger_provider.add_log_record_processor(BatchLogRecordProcessor(exporter))
handler = LoggingHandler(level=logging.NOTSET, logger_provider=logger_provider)

# Attach OTLP handler to root logger
logging.getLogger().addHandler(handler)

# Log directly
logging.info("Jackdaws love my big sphinx of quartz.")

# Create different namespaced loggers
logger1 = logging.getLogger("myapp.area1")
logger2 = logging.getLogger("myapp.area2")

logger1.debug("Quick zephyrs blow, vexing daft Jim.")
logger1.info("How quickly daft jumping zebras vex.")
logger2.warning("Jail zesty vixen who grabbed pay from quack.")
logger2.error("The five boxing wizards jump quickly.")

# Trace context correlation
tracer = trace.get_tracer(__name__)
with tracer.start_as_current_span("foo"):
    # Do something
    logger2.error("Hyderabad, we have a major problem.")

logger_provider.shutdown()

Resource SchemaURL is missing in logs and in spans:

2024-01-28T13:50:47.556Z        INFO    loggingexporter/logging_exporter.go:72  LogsExporter    {"#logs": 3}
2024-01-28T13:50:47.556Z        DEBUG   loggingexporter/logging_exporter.go:82  ResourceLog #0
Resource SchemaURL:
Resource labels:
     -> telemetry.sdk.language: STRING(python)
     -> telemetry.sdk.name: STRING(opentelemetry)
     -> telemetry.sdk.version: STRING(1.23.0.dev0)
     -> service.name: STRING(shoppingcart)
     -> service.instance.id: STRING(instance-12)
ScopeLogs #0
ScopeLogs SchemaURL:
InstrumentationScope opentelemetry.sdk._logs._internal
LogRecord #0
ObservedTimestamp: 1970-01-01 00:00:00 +0000 UTC
Timestamp: 2024-01-28 13:50:47.538600448 +0000 UTC
Severity: WARNING
Body: Jail zesty vixen who grabbed pay from quack.
Trace ID:
Span ID:
Flags: 0
LogRecord #1
ObservedTimestamp: 1970-01-01 00:00:00 +0000 UTC
Timestamp: 2024-01-28 13:50:47.551443968 +0000 UTC
Severity: ERROR
Body: The five boxing wizards jump quickly.
Trace ID:
Span ID:
Flags: 0
LogRecord #2
ObservedTimestamp: 1970-01-01 00:00:00 +0000 UTC
Timestamp: 2024-01-28 13:50:47.551934976 +0000 UTC
Severity: ERROR
Body: Hyderabad, we have a major problem.
Trace ID: 3ce9097d8505cf1319c235ce6c7101c8
Span ID: e2ae22ab654a3108
Flags: 1

2024-01-28T13:50:47.559Z        INFO    loggingexporter/logging_exporter.go:43  TracesExporter  {"#spans": 1}
2024-01-28T13:50:47.559Z        DEBUG   loggingexporter/logging_exporter.go:52  ResourceSpans #0
Resource SchemaURL:
Resource labels:
     -> telemetry.sdk.language: STRING(python)
     -> telemetry.sdk.name: STRING(opentelemetry)
     -> telemetry.sdk.version: STRING(1.23.0.dev0)
     -> service.name: STRING(shoppingcart)
     -> service.instance.id: STRING(instance-12)
ScopeSpans #0
ScopeSpans SchemaURL:
InstrumentationScope __main__
Span #0
    Trace ID       : 3ce9097d8505cf1319c235ce6c7101c8
    Parent ID      :
    ID             : e2ae22ab654a3108
    Name           : foo
    Kind           : SPAN_KIND_INTERNAL
    Start time     : 2024-01-28 13:50:47.551901613 +0000 UTC
    End time       : 2024-01-28 13:50:47.551996529 +0000 UTC
    Status code    : STATUS_CODE_UNSET
    Status message :
hirotasoshu commented 7 months ago

Yeah, it's missed in metrics too, gonna make a pr with fixes soon