Failure to disable Resource provider logs.

[AntonioLyubchev]

Describe the bug Failure to disable resource properties from a structured log message.

I want to log only resource attributes I set in the configuration YAML, so I added

otel:
  java:
    enabled:
      resource:
        providers: io.opentelemetry.sdk.autoconfigure.internal.EnvironmentResourceProvider

Which almost did the job, I still get:

"telemetry": {
  "sdk": {
    "name": "opentelemetry",
    "language": "java",
    "version": "1.42.1"
  }
}

I tried using:

otel:
  experimental:
    resource:
      disabled:
        keys: [ telemetry.sdk.language, telemetry.sdk.name, telemetry.sdk.version ] not working

Steps to reproduce Create a project using Spring Boot Starter and configure otlp export to collector through grpc. Produce a log message. Observe the log message in elastic through the elastic exporter.

What did you expect to see? When specifying io.opentelemetry.sdk.autoconfigure.internal.EnvironmentResourceProvider I expected to see only

"Resource": {
      "environment": "dev",
      "service": {
        "name": "PoC-OpenTelemetry-SP3-Webflux-REST"
      }
}

since this is what I have set in the application yaml.

What did you see instead? I saw an additional resource in the structured log message:

"telemetry": {
  "sdk": {
    "name": "opentelemetry",
    "language": "java",
    "version": "1.42.1"
  }
}

Full YAML config:

# Simulated environment variable injection
APP_ENV: dev
LOGBACK_EXPERIMENTAL_LOG_ATTRIBUTES: false # Enable the capture of experimental log attributes thread.name and thread.id
LOGBACK_CAPTURE_MDC_ATTRIBUTES: "*" # Comma separated list of MDC attributes to capture. Use the wildcard character * to capture all attributes.
LOGBACK_CAPTURE_CODE_ATTRIBUTES: false # Enable the capture of source code attributes. Note that capturing source code attributes at logging sites might add a performance overhead.
LOGBACK_CAPTURE_MARKER_ATTRIBUTE: false # Enable the capture of Logback markers as attributes.
LOGBACK_CAPTURE_LOGGER_CONTEXT_ATTRIBUTES: false  # Enable the capture of Logback logger context properties as attributes. (This feature adds MDC properties as top level attributes instead of nested in _source)
LOGBACK_CAPTURE_KEY_VALUE_PAIR_ATTRIBUTES: false  # Enable the capture of Logback key value pairs as attributes.
OTLP_PROTOCOL: grpc
OTLP_ENDPOINT: http://localhost:4317

otel:
  exporter:
    otlp:
      endpoint: ${OTLP_ENDPOINT}
      protocol: ${OTLP_PROTOCOL}
  metrics:
    exporter: [ otlp ]
  traces:
    exporter: [ otlp ]
  propagators: [ tracecontext ]
  resource:
    attributes:
      environment: ${APP_ENV}
  instrumentation:
    logback-appender:
      experimental-log-attributes: ${LOGBACK_EXPERIMENTAL_LOG_ATTRIBUTES}
      experimental:
        capture-mdc-attributes: ${LOGBACK_CAPTURE_MDC_ATTRIBUTES}
        capture-code-attributes: ${LOGBACK_CAPTURE_CODE_ATTRIBUTES}
        capture-marker-attribute: ${LOGBACK_CAPTURE_MARKER_ATTRIBUTE}
        capture-logger-context-attributes: ${LOGBACK_CAPTURE_LOGGER_CONTEXT_ATTRIBUTES}
        capture-key-value-pair-attributes: ${LOGBACK_CAPTURE_KEY_VALUE_PAIR_ATTRIBUTES}
  java:
    enabled:
      resource:
        providers: io.opentelemetry.sdk.autoconfigure.internal.EnvironmentResourceProvider
  experimental:
    resource:
      disabled:
        keys: [ telemetry.sdk.language, telemetry.sdk.name, telemetry.sdk.version ]
  service:
    name: ${spring.application.name}

What version and what artifacts are you using? How did you reference these artifacts? pom.xml

<dependency>
    <groupId>io.opentelemetry.instrumentation</groupId>
    <artifactId>opentelemetry-spring-boot-starter</artifactId>
</dependency>

...

<dependencyManagement>
    <dependencies>
        <dependency>
            <groupId>io.opentelemetry.instrumentation</groupId>
            <artifactId>opentelemetry-instrumentation-bom</artifactId>
            <version>2.8.0</version>
            <type>pom</type>
            <scope>import</scope>
        </dependency>
    </dependencies>
</dependencyManagement>

Environment Compiler: coretto-21.0.4 OS: Windows 11

Additional context Just to note: io.opentelemetry.sdk.autoconfigure.internal.EnvironmentResourceProvider is not described here: https://opentelemetry.io/docs/languages/java/configuration/#resourceprovider

Also, can you please tell me how can I toggle all the other attributes present in the structured log: For example, I don't want:

• TraceFlags
• SeverityText
• SeverityNumber
• data_stream.type

[jack-berg]

Oops replied here because I saw that comment before this issue:

Let's use this as an issue to track fixing otel.experimental.resource.disabled-keys to apply to the default attributes included in Resource.getDefault()

[AntonioLyubchev]

@jack-berg I see that the disabled keys option is labeled: "This option is experimental and subject to change or removal." But I think this is incredibly important to be present since we need to control what's added to the structured log.

[AntonioLyubchev]

@jack-berg ahh.... found it

The yaml format is restricted to comma separated list. Found it after some debugging and digging:

This solves the issue with the SDK attributes:

otel:
  experimental:
    resource:
      disabled:
        keys: "telemetry.sdk.language,telemetry.sdk.name,telemetry.sdk.version,data_stream.type,SeverityNumber"

However, SeverityNumber and data_stream.type are still present in the final log.

I assume it's because the functionality removes only "Resources", but in that case how do we remove other unnecessary fields?

On a side note, can I log somehow the JSON message to the console for debugging purposes? Since the collector's "debug" exporter doesn't show the literal JSON message in the output.

[jack-berg]

But I think this is incredibly important to be present since we need to control what's added to the structured log.

I agree. All of our properties go through a maturation cycle where they start as experimental for a period of time before being promoted to stable, with all the strong backwards compatibility guarantees. otel.experimental.resource.disabled-keys has been around for a while and its probably time to consider stabilizing it.

However, SeverityNumber and data_stream.type are still present in the final log. I assume it's because the functionality removes only "Resources", but in that case how do we remove other unnecessary fields?

SeverityNumber is a top level field in the log data model, so resource attribute configuration has no affect. If you're using the collector, you may be able to remove it with the transformprocessor (haven't confirmed this but its what I would try).

I'm not sure what data_stream.type is. It may be some artifact of the serializer used to print the data.

On a side note, can I log somehow the JSON message to the console for debugging purposes? Since the collector's "debug" exporter doesn't show the literal JSON message in the output.

You may be interested in the collector's fileexporter, which prints the OTLP JSON encoding of messages to files.

In opentelemetry-java we publish OtlpJsonLogging{Signal}Exporters out of io.opentelemetry:opentelemetry-exporter-logging-otlp. These are configured by setting OTEL_LOGS_EXPORTER=logging-otlp, as disusssed here.

[AntonioLyubchev]

Closing as there is no more work here.

A good idea would be documenting that most yaml properties strictly rely on comma separated list.

[jack-berg]

A good idea would be documenting that most yaml properties strictly rely on comma separated list.

The YAML format you reference isn't defined as part of this repo. I believe its specific to the spring boot autoconfigure module. Maybe @zeitlinger, @jeanbisutti can comment.

[zeitlinger]

the spring starter does support native lists - if proper support is added here - I simply forgot that one

[zeitlinger]

Here's the PR to fix this https://github.com/open-telemetry/opentelemetry-java-instrumentation/pull/12434