More programmatic control of agent parameters from the app itself.

[anuraaga]

Users do want to be able to hook in existing configuration systems to telemetry.

https://github.com/open-telemetry/opentelemetry-java-instrumentation/discussions/1998

I'm glad the SDK has made good steps in supporting this better but unfortunately the agent does still follow a model of no programmatic control from apps. It means users cannot develop frameworks to control telemetry - they must develop custom agent distros.

If it's possible to develop a way for a user app classloader to influence telemetry configuration, it would be a great improvement I think.

/cc @tobias-

[anuraaga]

One way to put it

I do fully expect telemetry vendors to be able to create custom agent distros. This requires significant knowledge of complicated topics like classloaders and shading.

I don't expect users to do so though. But ideally we can let them configure an exporter auth key the same way they configure a cloud SDK client, which may be something like lightbend config or spring boot. After writing this sentence I wondered whether the biggest hammer for this problem is to have instrumentation for all the popular config systems out there. Probably not though :)

[tobias-]

I think I've said most of this in other threads, but I'll try to summarize the parts about programmatic control (with the usual caveat that I've just been doing it wrong).

I worked around the problem with getting the exporter (by just using the google offering), but even if I can imagine someone writing their own exporter for a special format, for dumping into database whatnot, I don't believe many people will.

That said, getting access to programmatically changing the configuration would be really useful for what I believe to be a majority. Many posts #1950, #1958, #1060, #2003, #1962 are about controlling what gets sampled, so just adding the ability to modify the config would be a lot gained. If combined with the ability to set a 'noop' exporter with e.g. -Dotel.exporter=noop, even adding your own exporter would be possible. No idea if that's possible though.

For now I'll hack in the Sampler or filtering via the gcp driver, since I need to compile that from source anyway.

One idea:

1. User creates a config class (class annotated with e.g. @TraceConfig or extending an e.g. TraceInstrumentationConfig).
2. The agent loads it into its own class loader (yes, I understand that Kodein and other DI would be impossible)
3. The class contains methods which are hooks that can e.g. decide sampling based on completely arbitrary parameters. E.g. my example in #2003 where I need to switch on thread name.

This will not allow the app (the instrumented code) to add exporters, but just by having control over the Sampler would probably solve 90% of the configuration problems. The solution would most likely not be really 'kosher', as the class path you're compiling against will not be the runtime class path, but maybe a huge caveat notice in the extended/annotation class, and a good error message if the class loading fails, could suffice?

If you want to get fancy with the idea, you could add some rudimentary DI-like functionality such as allowing multiple classes with invocation priority and pass-through for methods.

I have no idea what would work, but another similar idea would be to have a "hook"-class for every instrumenter that could be used like (in the instrumented code):

@InstrumentationHook(instrumenter = "io.opentelemetry.armeria")
class NoHealth extends TracerHook {
    @Override
    SamplingResult hook(...) {
        if (ServiceRequestContext.current().req().path.startsWith("/health") {
            return create(DROP);
        }
        return FALL_THROUGH;
    }
}

[chenmudu]

@anuraaga @tobias- Perhaps we can take another approach to configuring discovery, the simplest and most efficient one is to allow users to configure it locally and place it in a file.Configure the path of the file in the system boot parameter.I don't know how you feel about that. Thanks a lot.

[anuraaga]

@chenmudu We do support reading agent parameters from a file already - it can work well if people use an otel-specific config file. However, the programmatic control would benefit so users can use their existing config system - for example, if a user uses application-*.yml files for spring boot config, it would be very natural if this can control OpenTelemetry too.

[chenmudu]

@anuraaga

Thank you for your reply. It seems that I have not found the document specification similar to Otel-specific. Could you please give me a Web link? Thanks a lot.

I quite agree with your idea.

I choose the intrusive SDK rather than the Agent enhancement in my work content, although the non-intrusive Agent is more in line with the enterprise-level development, largely because I hope to provide convenient and simple configurable content for users.I wonder if it is possible to provide a syntax based on Java itself to do similar extensions.

Personally, I think it is too difficult and not very convenient to do extensions for every intrusive SDK (eg.SpringBoot: we may need to provide the ability of automatic configuration for it, and then the agent side needs to scan and find it. However, the order of the two loads seems to be a bit out of sync, I mean that the otel-agent loads as if the Spring container has not been started.).

If you think it is feasible, could you please introduce it to me in a little detail? I am a novice in this field, and I would appreciate it very much. Maybe there are other possibilities?

Looking forward to your and other's reply. Thank you very much.