search

confluentinc / ide-sidecar

Sidecar application used by Confluent for VS Code, as a local proxy for Confluent Cloud, Confluent Platform and local Kafka clusters, to help users build streaming applications.
Apache License 2.0
3 stars 3 forks source link

test ClusterStrategyTest#[4] flaking #146

Closed Cerchie closed 6 days ago

Cerchie commented 1 week ago

To (possibly) reproduce, try cloning the repo and running make quarkus-dev

Once, I got this. 

Running 141/314. Running: io.confluent.idesidecar.restapi.proxy.clusters.strategy.ClusterStrategyTest#[4] proxyResponse={
2024-11-06 12:51:42,785 ERROR [io.qua.test] (Test runner thread) Test ConfluentCloudQueryResourceTest#shouldCacheClusterInfoOnGetKafkaClusters() failed 
: io.confluent.idesidecar.restapi.exceptions.ClusterNotFoundException: Kafka Cluster 4L6g3nShT-eMCtK--X86sw not found in connection local-1
        at io.confluent.idesidecar.restapi.cache.ClusterCache$Clusters.findKafkaCluster(ClusterCache.java:398)
        at io.confluent.idesidecar.restapi.cache.ClusterCache$Clusters.getKafkaCluster(ClusterCache.java:295)
        at io.confluent.idesidecar.restapi.cache.ClusterCacheAssertions.assertKafkaClusterCached(ClusterCacheAssertions.java:50)
        at io.confluent.idesidecar.restapi.cache.ClusterCacheAssertions.assertKafkaClusterCached(ClusterCacheAssertions.java:39)
2024-11-06 12:51:42,788 ERROR [io.qua.test] (Test runner thread) >>>>>>>>>>>>>>>>>>>> Summary: <<<<<<<<<<<<<<<<<<<<kaClusters(ConfluentCloudQueryResourceTest.java:236)
io.confluent.idesidecar.restapi.resources.ConfluentCloudQueryResourceTest#shouldCacheClusterInfoOnGetKafkaClusters(ConfluentCloudQueryResourceTest.java:236) ConfluentCloudQueryResourceTest#shouldCacheClusterInfoOnGetKafkaClusters() Kafka Cluster 4L6g3nShT-eMCtK--X86sw not found in connection local-1
2024-11-06 12:51:42,789 ERROR [io.qua.test] (Test runner thread) >>>>>>>>>>>>>>>>>>>> 1 TEST FAILED <<<<<<<<<<<<<<<<<<<<

Didn't change anything, re-ran the tests with quarkus, and everything passed.

rohitsanj commented 1 week ago

The error logs might be misleading. The ClusterStrategyTest and ConfluentCloudQueryResourceTest#shouldCacheClusterInfoOnGetKafkaClusters are separate tests.

As for the shouldCacheClusterInfoOnGetKafkaClusters failure, there's an annotation that marks it as @ExpectedToFail because we've seen it be flaky in the past. See: https://github.com/confluentinc/ide-sidecar/blob/0d355a0334e0d525968faf2960a512a9b73f9da1/src/test/java/io/confluent/idesidecar/restapi/resources/ConfluentCloudQueryResourceTest.java#L186-L188

Edit: Now that I read the comment, we're not actually retrying three times. That comment needs an update.

Cerchie commented 6 days ago

Thanks for the update, will close