Feature Test Summary for MpTelemetry-2.0

[yasmin-aumeeruddy]

Test Strategy

List of FAT projects affected

• io.openliberty.microprofile.telemetry.1.0.internal_fat
• io.openliberty.microprofile.telemetry.1.0.internal.container_fat
• io.openliberty.http.monitor_fat
• io.openliberty.microprofile.telemetry.2.0.logging.internal_fat (NEW)
• io.openliberty.microprofile.telemetry.2.0.logging.internal.container_fat (NEW)
• io.openliberty.microprofile.telemetry.2.0.internal_fat_tck (NEW)

Test strategy

External open source packages have been updated from OpenTelemetry version 1.29.0 to 1.39.0

The feature is compatible with older MicroProfile versions and older Java EE versions (7+) . Therefore, Repeats are required for all of the aforementioned projects for MpTelemetry-1.1 with the following MicroProfile versions:

• MP 1.4 (EE 7)
• MP 4.1 (EE 8)
• MP 5.0 (EE 9)
• MP 7.0 (EE 10)
• MP 7.0 (EE 11)

Any test that currently runs against mpTelemetry-1.1 should also work with mpTelemetry-2.0. However, test repeats for mpTelemetry 2.0 should test the updated SemanticAttributes class . The class has changed package from io.opentelemetry.semconv.trace.attributes to io.opentelemetry.semconv In addition,Jaeger is now a deprecated trace exporter confiuration value so tests for this exporter option should not be repeated for mpTelemetry-2.0 (Telemetry data is sent via OTLP to a jaeger backend instead).

Tests have been created for:

• Behaviour as a result of the Java Agent with the 1.39 version (Agent210Test)
• The OpenTracing Shim with the 1.39 version (TelemetryShimTest)
• Ensuring that the OpenTelemetry instance can be initilized at runtime startup with the appropriate configuration (system properties and environment variables) (TelemetryRuntimeInstanceTest)
• Ensuring that the OpenTelemetry instance can be initilized at application startup with the appropriate configuration (accessed via mpConfig except for system properties and environment variables)
• Ensuring JVM metrics are sent correctly via otlp (JvmMetricsOtelCollectorTest)
• Ensuring JVM metrics are sent correctly via the logging exporter (Metrics JVM TCK)
• Ensuring HTTP metrics are sent correctly via otlp and the logging exporter
• Ensuring mannually created metrics are sent correctly via otlp
• Ensuring mannually created metrics are sent correctly via the logging exporter (Http TCK)
• Ensure that all logs can be collected by OpenTelemetry and sent correctly via otlp and logging. Including:
  • Messages
  • Trace
  • FFDC

MpTelemetry 2.0 is compatible with Java 8. The following buckets repeat tests with this Java version:

• io.openliberty.microprofile.telemetry.1.0.internal_fat
• io.openliberty.microprofile.telemetry.1.0.internal.container_fat
• io.openliberty.http.monitor_fat
• io.openliberty.microprofile.telemetry.2.0.logging.internal_fat (NEW)
• io.openliberty.microprofile.telemetry.2.0.logging.internal.container_fat (NEW)

Confidence score:

4 - We have delivered all automated testing we believe is needed for the golden paths of this feature and have good coverage of the error/outlying scenarios. While more testing of the error/outlying scenarios could be added we believe there is minimal risk here and the cost of providing these is considered higher than the benefit they would provide.

[benjamin-confino]

Currently some of the new features are explicitly tested in both runtime and app mode, but a lot of the old stuff is tested in whatever way the configuration enables (usually runtime).

We need to setup a system where every test runs on both, then carve out exceptions for the ones that are only supposed to work in one instance.

[benjamin-confino]

Myself and Yasmin went through and identified all areas we need new tests to ensure we're covered on both runtime and appmode. I am now confident on that issue.

[dave-waddling]

Hi, thanks for completing the FTS, the content passes review. However, until RTC:301434 is resolved I can't grant FAT Focal approval.

[dave-waddling]

@yasmin-aumeeruddy based on our Slack discussion I'm willing to grant approval at this time, I'll add it to the feature issue now.