search

quarkus-qe / quarkus-test-suite

16 stars 34 forks source link

[3.15] Disable SSL and SASL_SSL Kafka and Infinispan tests in FIPS and native mode as these scenarios are currently not supported #2136

Closed michalvavrik closed 1 month ago

michalvavrik commented 1 month ago

Summary

FIPS in native are currently not supported, we have product trackers to alert us if anything changes, for now we need to disable tests due to failures experienced in the rhbq-3.15-baremetal-ts-native Jenkins job.

Please select the relevant options.

Checklist:

michalvavrik commented 1 month ago

Thank you! But given the situation, I would suggest to merge to main with backport label (I'll do a backport together with #2135)

I have no plan to disable it in main. Who knows what will be in half of the year? I cannot set affected version to Jumanji until I know it will fail with it. And I don't know if this will be still case for the next Mandrel or what feedback we get - like what if feedback is we need to rewrite it?

michalvavrik commented 1 month ago

Also if I remember correctly this annotation is not present in current main, so I can do that, but it will require FW release so you will need to wait few hours.

michalvavrik commented 1 month ago

@fedinskiy I don't really mind, if that is what you prefer, ok, but I cannot provide fix before FW is released or I add some unnecessary repetition. Again, I don't mind either of that, so I'll let you decide.

github-actions[bot] commented 1 month ago

Unknown format for flaky report filename: flaky-run-report.json Following jobs contain at least one flaky test:

Run summary: https://github.com/quarkus-qe/quarkus-test-suite/actions/runs/11518377394?pr=2136

Flaky tests:

io.quarkus.ts.messaging.kafka.reactive.streams.KafkaGratefulShutdownIT.testConnection

org.awaitility.core.ConditionTimeoutException: Assertion condition defined as a Lambda expression in io.quarkus.test.utils.LogsVerifier Log does not contain any of '[ave]'. Full logs: [__  ____  __  _____   ___  __ ____  ______ ,  --/ __ \/ / / / _ | / _ \/ //_/ / / / __/ ,  -/ /_/ / /_/ / __ |/ , _/ ,< / /_/ /\ \   , --\___\_\____/_/ |_/_/|_/_/|_|\____/___/   , 13:10:52,671 SRMSG18229: Configured topics for channel 'slow': [slow], 13:10:52,686 SRMSG18214: Key deserializer omitted, using String as default, 13:10:52,686 SRMSG18234: Auto-commit disabled for channel slow, 13:10:52,851 SRMSG18240: 'throttled' commit strategy used for channel 'slow', 13:10:52,852 SRMSG18232: Will commit for group 'kafka-streams-reactive-messaging' every 5000 milliseconds., 13:10:52,852 SRMSG18236: Will mark throttled commit strategy for group 'kafka-streams-reactive-messaging' as unhealthy if records go more than 60000 milliseconds without being processed., 13:10:52,881 SRMSG18248: Key serializer omitted, using String as default, 13:10:52,907 SRMSG18258: Kafka producer kafka-producer-slow-topic, connected to Kafka brokers 'localhost:32777', is configured to write records to 'slow', 13:10:52,950 SRMSG18257: Kafka consumer kafka-consumer-slow, connected to Kafka brokers 'localhost:32777', belongs to the 'kafka-streams-reactive-messaging' consumer group and is configured to poll records from [slow], 13:10:52,957 No Topology producer; Kafka Streams will not be started, 13:10:53,050 kafka-streams-reactive-messaging 1.0.0-SNAPSHOT on JVM (powered by Quarkus 3.15.999-SNAPSHOT) started in 1.326s. Listening on: http://0.0.0.0:1104, 13:10:53,050 Profile prod activated. , 13:10:53,050 Installed features: [cdi, kafka-client, kafka-streams, messaging, messaging-kafka, resteasy, resteasy-jsonb, smallrye-context-propagation, vertx], 13:10:53,177 [Consumer clientId=kafka-consumer-slow, groupId=kafka-streams-reactive-messaging] Error while fetching metadata with correlation id 2 : {slow=UNKNOWN_TOPIC_OR_PARTITION}, 13:10:56,666 SRMSG18209: Sending message org.eclipse.microprofile.reactive.messaging.Message$5@7496c7e2 from channel 'slow-topic' to Kafka topic 'slow', 13:10:56,766 SRMSG18211: Message org.eclipse.microprofile.reactive.messaging.Message$5@7496c7e2 from channel 'slow-topic' sent successfully to Kafka topic-partition 'slow-0', with offset 0] ==> expected: <true> but was: <false> within 30 seconds.
    at org.awaitility.core.ConditionAwaiter.await(ConditionAwaiter.java:167)
    at org.awaitility.core.AssertionCondition.await(AssertionCondition.java:119)
    at org.awaitility.core.AssertionCondition.await(AssertionCondition.java:31)
    at org.awaitility.core.ConditionFactory.until(ConditionFactory.java:1006)
    at org.awaitility.core.ConditionFactory.untilAsserted(ConditionFactory.java:790)
    at io.quarkus.test.utils.AwaitilityUtils.untilAsserted(AwaitilityUtils.java:134)
    at io.quarkus.test.utils.AwaitilityUtils.untilAsserted(AwaitilityUtils.java:124)
    at io.quarkus.test.utils.LogsVerifier.assertContains(LogsVerifier.java:28)
    at io.quarkus.ts.messaging.kafka.reactive.streams.KafkaGratefulShutdownIT.testConnection(KafkaGratefulShutdownIT.java:72)
    at java.base/java.lang.reflect.Method.invoke(Method.java:569)
    at java.base/java.util.ArrayList.forEach(ArrayList.java:1511)
    at java.base/java.util.ArrayList.forEach(ArrayList.java:1511)
Caused by: org.opentest4j.AssertionFailedError: Log does not contain any of '[ave]'. Full logs: [__  ____  __  _____   ___  __ ____  ______ ,  --/ __ \/ / / / _ | / _ \/ //_/ / / / __/ ,  -/ /_/ / /_/ / __ |/ , _/ ,< / /_/ /\ \   , --\___\_\____/_/ |_/_/|_/_/|_|\____/___/   , 13:10:52,671 SRMSG18229: Configured topics for channel 'slow': [slow], 13:10:52,686 SRMSG18214: Key deserializer omitted, using String as default, 13:10:52,686 SRMSG18234: Auto-commit disabled for channel slow, 13:10:52,851 SRMSG18240: 'throttled' commit strategy used for channel 'slow', 13:10:52,852 SRMSG18232: Will commit for group 'kafka-streams-reactive-messaging' every 5000 milliseconds., 13:10:52,852 SRMSG18236: Will mark throttled commit strategy for group 'kafka-streams-reactive-messaging' as unhealthy if records go more than 60000 milliseconds without being processed., 13:10:52,881 SRMSG18248: Key serializer omitted, using String as default, 13:10:52,907 SRMSG18258: Kafka producer kafka-producer-slow-topic, connected to Kafka brokers 'localhost:32777', is configured to write records to 'slow', 13:10:52,950 SRMSG18257: Kafka consumer kafka-consumer-slow, connected to Kafka brokers 'localhost:32777', belongs to the 'kafka-streams-reactive-messaging' consumer group and is configured to poll records from [slow], 13:10:52,957 No Topology producer; Kafka Streams will not be started, 13:10:53,050 kafka-streams-reactive-messaging 1.0.0-SNAPSHOT on JVM (powered by Quarkus 3.15.999-SNAPSHOT) started in 1.326s. Listening on: http://0.0.0.0:1104, 13:10:53,050 Profile prod activated. , 13:10:53,050 Installed features: [cdi, kafka-client, kafka-streams, messaging, messaging-kafka, resteasy, resteasy-jsonb, smallrye-context-propagation, vertx], 13:10:53,177 [Consumer clientId=kafka-consumer-slow, groupId=kafka-streams-reactive-messaging] Error while fetching metadata with correlation id 2 : {slow=UNKNOWN_TOPIC_OR_PARTITION}, 13:10:56,666 SRMSG18209: Sending message org.eclipse.microprofile.reactive.messaging.Message$5@7496c7e2 from channel 'slow-topic' to Kafka topic 'slow', 13:10:56,766 SRMSG18211: Message org.eclipse.microprofile.reactive.messaging.Message$5@7496c7e2 from channel 'slow-topic' sent successfully to Kafka topic-partition 'slow-0', with offset 0] ==> expected: <true> but was: <false>
    at io.quarkus.test.utils.LogsVerifier.lambda$assertContains$0(LogsVerifier.java:30)
    at org.awaitility.core.AssertionCondition.lambda$new$0(AssertionCondition.java:53)
    at org.awaitility.core.ConditionAwaiter$ConditionPoller.call(ConditionAwaiter.java:248)
    at org.awaitility.core.ConditionAwaiter$ConditionPoller.call(ConditionAwaiter.java:235)
    at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
    at java.base/java.lang.Thread.run(Thread.java:840)

fedinskiy commented 1 month ago

@michalvavrik we are not in a hurry, so waiting for several hours (or even days) is not a problem. Since this is not fixed in any version of Quarkus/Mandrel (yet) and given a speed these bugs are resolved with[1], we can reasonably expect, that this error will affect Jumanji, and possibly even releases after that. I would like to save us some time from investigation of these failures and disable it now. Ideally, also post a link to this PR to these issues, so when they are fixed, we can enable/adjust these tests as well

[1] eg https://issues.redhat.com/browse/QUARKUS-4612

michalvavrik commented 1 month ago

sure @fedinskiy