Add support to OTEL_RESOURCE_ATTRIBUTES in the SDK without relaying on SPI

[rapphil]

Is your feature request related to a problem? Please describe.

The spec defines that the SDK must extract information from the OTEL_RESOURCE_ATTRIBUTES environment variable.

However io.opentelemetry.sdk.resources.Resource does not honor OTEL_RESOURCE_ATTRIBUTES. The equivalent of this class in other languages does honor that. E.g.:

• Python
• Golang

This functionality is only available in the Autoconfigure SDK, which requires that you use SPI.

Describe the solution you'd like

I would like to be able to use the regular SDK and programmatically detect resource attributes from environment variables and optionally system properties.

Something like:

Resource resource = Resource.getDefault().merge(EnvironmentResource.get())

Describe alternatives you've considered I haven't considered any other alternatives.

Additional context

The current logic to detect resource attributes from the OTEL_RESOURCE_ATTRIBUTES env var could be extracted out of the Autoconfigure SDK and implemented somewhere else (in the SDK or other package - TBD). The Autoconfigure SDK would then rely on it. This could be potentially applied to all resources that are configurable through environment variables.

However, It might not be simple to extract this functionality of the Autoconfigure SDK because this interface is a public interface https://github.com/open-telemetry/opentelemetry-java/blob/main/sdk-extensions/autoconfigure-spi/src/main/java/io/opentelemetry/sdk/autoconfigure/spi/ConfigProperties.java

[jkwatson]

What if we just included an EnvironmentResource as a part of the autoconfigure module? Then we wouldn't have any dependency issues, since that module already depends on the SPI module for the ConfigProperties class.

[jack-berg]

This functionality is only available in the Autoconfigure SDK, which requires that you use SPI.

Can you clarify what you mean by "requires that you use the SPI"? The code that reads OTEL_RESOURCE_ATTRIBUTES is implemented in ResourceConfiguration, which is not an implementation of the ResourceProvider SPI.

[rapphil]

@jack-berg

Can you clarify what you mean by "requires that you use the SPI"? The code that reads OTEL_RESOURCE_ATTRIBUTES is implemented in ResourceConfiguration, which is not an implementation of the ResourceProvider SPI.

ResourceConfiguration tries to load classes that implement ResourceProvider here: https://github.com/open-telemetry/opentelemetry-java/blob/main/sdk-extensions/autoconfigure/src/main/java/io/opentelemetry/sdk/autoconfigure/ResourceConfiguration.java#L47 , even if you don't have any classes implementing ResourceProvider or you don't want to try to dynamically load classes due to various concerns.

I wish there was a version of AutoConfiguredOpenTelemetrySdk that by default only tries to configure components using environment variables. More advanced stuff could be provided through customizers passed as parameter to any of the add*Customizer methods.

@jkwatson

What if we just included an EnvironmentResource as a part of the autoconfigure module? Then we wouldn't have any dependency issues, since that module already depends on the SPI module for the ConfigProperties class.

I think this is a good trade off. I will create a proposal using this approach.

[jack-berg]

You can effectively achieve that result today with something like:

AutoConfiguredOpenTelemetrySdk.builder()
        .setServiceClassLoader(new ClassLoader() {}) // Disable SPI class loading by specifying custom class loader that doesn't detect SPI implementations
        .setResultAsGlobal(false) // Disable GlobalOpenTelemetry side affect
        .addPropertiesSupplier(() -> Collections.singletonMap("otel.sdk.disabled", "true")) // Disable sdk, so only resource gets autoconfigured
        .build()
        .getResource(); // Grab the resource

Not pretty, but it works.

Out of curiosity, why do you want a resource configured from OTEL_RESOURCE_ATTRIBUTES without the rest of the SDK configured with it?

[jkwatson]

@jack-berg I also need this, since I don't have the time to figure out how to finagle a bunch of custom, non-standard stuff in my SDK configuration along with auto-configuration. I literally only need a tiny piece of auto-configuration, and that's the resources. :)

Just for clarity, what I configure right now for the SDK we use at Verta is:

• Jaeger grpc exporter
• Metrics using the contrib module I maintain for the prometheus bridge
• A Resource built on a couple of custom attributes and a few of the resource providers in the instrumentation repo.

It might be possible to finagle this with autoconfigure, but aside from the jaeger bits, it's pretty much custom, so hasn't seemed like it would be worth the effort. :)

[rapphil]

Out of curiosity, why do you want a resource configured from OTEL_RESOURCE_ATTRIBUTES without the rest of the SDK configured with it?

I came across this because I was trying to manually instrument an application. I was trying to replicate the results obtained when automatically instrumenting it using the Java Agent. I took for granted that the OTEL_RESOURCE_ATTRIBUTES would be honored. The reason for my assumption is because of the statements in this section. I later realized that I was wrong.

SDKs MAY choose to allow configuration via the environment variables in this specification, but are not required to. If they do, they SHOULD use the names listed in this document.

Having said that, In general I think it would be useful to provide a common mechanism for extracting the OTEL_RESOURCE_ATTRIBUTES for folks that want to exclusively use the regular SDK:

• OTEL_RESOURCE_ATTRIBUTES feels ubiquitous. We use it to configure SDK with information that is environment dependent and that will be applied to the telemetry signals.e.g.: stage(production/test), region. The rest of the configuration of the SDK feels more hard coded (Span Processor/Exporter etc).
• Parsing OTEL_RESOURCE_ATTRIBUTES is not simple and we don't need to re implement it (not simple as reading a string/number from an env var).

[jack-berg]

@jkwatson to each his own, but I prefer to use autoconfigure in basically every situation, even with high customized configurations. The programatic configurability via AutoConfiguredOpenTelemetrySdkBuilder and the SPIs is pretty exhaustive, its nice to not have to reimplement reading config from the environment each time you want to modify the behavior of some component at runtime. I.e. its surprising when you try to use the documented environment variables and they don't work.

My concern is that there are many combinations of an autoconfigured-ish resource that a user might want. Here are the "base" sources of resources and customization:

• The environment resource sourced from OTEL_RESOURCE_ATTRIBUTES
• The SPI resource(s) sourced from ResourceProvider SPI
• Customized version of environment / SPI resource, via AutoConfiguredOpenTelemetrySdk.builder().addResourceCustomizer(...)
• Customized version of environment / SPI resource, via OTEL_JAVA_[ENABLED|DISABALED]_RESOURCE_PROVIDERS, OTEL_EXPERIMENTAL_RESOURCE_DISABLED_KEYS

These options can be combined in many ways based on unique requirements of a particular user's use case. For example, maybe a user wants the environment resource and some of the SPI resource, but wants to strip away a particular key via OTEL_EXPERIMENTAL_RESOURCE_DISABLED_KEYS, and wants to add their own attribute programmatically via addResourceCustomizer(..).

This request is essentially asking for a dedicated helper function for one specific source of a resource. Is the environment based resource more important, or more commonly used in a standalone fashion than the variations of a resource users may want? I'm not convinced. I think that adding a one-off API to accommodate it sets a strange precedence that will make it hard reject future requests for dedicated APIs for specific bits of autoconfiguration.

[jack-berg]

Resolved in #5554.