Auto-instrumentation with runtime-telemetry Java17

[sarkisjad]

Describe the bug

I am trying to generate metrics with the RuntimeMetrics class using JFR features. As per the documentation and the codeline, we should be able to generate metrics associated with only the enabled JFR Features by default. However, regardless of the flags added in the run command, I get all the metrics available, not the ones that are enabled by default.

Steps to reproduce

Run the command on the Windows terminal:

java -javaagent:./opentelemetry-javaagent.jar \
    -Dotel.instrumentation.runtime-telemetry-java17.enabled=true \
    -jar my-application.jar

Expected behavior

The emitted metrics should be the following:

• process.runtime.jvm.cpu.context_switch
• process.runtime.jvm.cpu.limit
• process.runtime.jvm.cpu.longlock
• process.runtime.jvm.memory.allocation
• process.runtime.jvm.network.io
• process.runtime.jvm.network.time

This is because these metrics are associated the enabled JfrFeatures by default which are:

• CONTEXT_SWITCH_METRICS
• CPU_COUNT_METRICS
• LOCK_METRICS
• MEMORY_ALLOCATION_METRICS
• NETWORK_IO_METRICS

Actual behavior

I get the metrics of all the JfrFeature, including the ones that are defaulted to false

Javaagent or library instrumentation version

v1.29.0 (latest as of date)

Environment

JDK: 17.0.2 OS:

Additional context

I have opentelemetry configured. It is listening to the default port (4317) and successfully capturing metrics and logging it in a random text file, as per my configuration. We have no issues on that side.

[mateuszrzeszutek]

Hey @sarkisjad ,

The current implementation of the JFR metrics only disables the metrics that are already produced by the JMX-based (Java 8 compatible) implementation; and there is no separate property for disabling the JMX based metrics in the javaagent. Can you check the instrumentation scope name of the metrics you're receiving? The ones generated by the JFR implementation should use io.opentelemetry.runtime-telemetry-java17; while the JMX-based metrics should use io.opentelemetry.runtime-telemetry-java8

[sarkisjad]

Thank you for your response @mateuszrzeszutek. Indeed, you are right. The metrics generated by the JFR implementation do have the scope io.opentelemetry.runtime-telemetry-java17 while the others have java8.

There was also a question I'd like to ask. In order to enable or disable features, is there a way to do so directly from the command line as opposed to having to write code? i.e. is there a way to do the following (from the documentation)

RuntimeMetrics runtimeMetrics = RuntimeMetrics.builder(openTelemetry)
  .enableFeature(JfrFeature.BUFFER_METRICS)
  .disableFeature(JfrFeature.LOCK_METRICS)
  .build();

Directly from the command line?

[mateuszrzeszutek]

No, currently there's no such configuration. You can do that when you use the library RuntimeMetrics instrumentation.

[sarkisjad]

So when I use the library in my code, is that how it should be done? My code simply adds values to a list of strings continuously until I terminate the program. It is a dummy application I use to play around with runtime metrics. Making use of the RuntimeMetrics library like in the doc does not remove the disabled JFR features that I specified. my-application.zip

[mateuszrzeszutek]

You're still running your app with the agent, right? First of all, you need to disable the metrics instrumentation bundled with the agent (-Dotel.instrumentation.runtime-telemetry.enabled=false); you'll be replacing it with your own use of RuntimeMetrics after all. Then you can enable/disable whatever you want in your own app (you can use GlobalOpenTelemetry to use the agent's SDK pipeline, but I can see you're already doing that).

[sarkisjad]

If I use the code that I wrote and run the following command: java -javaagent:./opentelemetry-javaagent.jar -Dotel.instrumentation-runtime-telemetry.enabled=false -Dotel.instrumentation.runtime-telemetry-java17.enabled=true -jar target/auto-instrumentation-1.0-SNAPSHOT-jar-with-dependencies.jar

The auto-instrumentation.zip application still does not filter out the JFR feature LOCK_METRICS

[mateuszrzeszutek]

Yes, you need to remove the -Dotel.instrumentation.runtime-telemetry-java17.enabled=true property as well. Since you're using the RuntimeMetrics manually in your own code, you need to turn off the same instrumentation in the javaagent; otherwise you're gonna get duplicate or conflicting telemetry.

[sarkisjad]

I still don't get any metrics after I remove -Dotel.instrumentation.runtime-telemetry-java17.enabled=true Is the code correct?