helidon-io / helidon

Java libraries for writing microservices
https://helidon.io
Apache License 2.0
3.5k stars 566 forks source link

4.x: Re-enable intermittently failing `ExemplarTest` #7796

Open tomas-langer opened 11 months ago

tomas-langer commented 11 months ago

io.helidon.metrics.exemplartrace.ExemplarTest#checkForExemplarsInOpenMetricsOutput

From first look this seems to be a race condition.

Expected:

>> skip to counter >>
testCounter_total.*?#.*?span_id=.*
>> end of output >>

Actual:

# TYPE requests_count counter
# HELP requests_count Each request (regardless of HTTP method) will increase this counter
requests_count_total{scope="vendor"} 2.0 # {span_id="0000000000000000",trace_id="00000000000000000000000000000000"} 1.0 1697317429.159
# TYPE testCounter counter
# HELP testCounter
testCounter_total{scope="application"} 1.0 # {span_id="0000000000000000",trace_id="00000000000000000000000000000000"} 1.0 1697317429.164

Failure:

org.opentest4j.AssertionFailedError: Counter output with exemplar ==> expected line #3:`>> end of output >>` not found - actual lines depleted
    at org.junit.jupiter.api.AssertionFailureBuilder.build(AssertionFailureBuilder.java:151)
    at org.junit.jupiter.api.AssertionFailureBuilder.buildAndThrow(AssertionFailureBuilder.java:132)
    at org.junit.jupiter.api.AssertLinesMatch$LinesMatcher.fail(AssertLinesMatch.java:203)
    at org.junit.jupiter.api.AssertLinesMatch$LinesMatcher.assertLinesMatchWithFastForward(AssertLinesMatch.java:124)
    at org.junit.jupiter.api.AssertLinesMatch$LinesMatcher.assertLinesMatch(AssertLinesMatch.java:112)
    at org.junit.jupiter.api.AssertLinesMatch.assertLinesMatch(AssertLinesMatch.java:80)
    at org.junit.jupiter.api.AssertLinesMatch.assertLinesMatch(AssertLinesMatch.java:46)
    at org.junit.jupiter.api.Assertions.assertLinesMatch(Assertions.java:1614)
    at io.helidon.metrics.exemplartrace.ExemplarTest.checkForExemplarsInOpenMetricsOutput(ExemplarTest.java:80)
jbescos commented 4 months ago

In my laptop I executed the test 327 times and it worked always.

Maybe we can enable it again and re-open it in case we find a failure.