ISSUE-13299: Flaky tests in branch-2.7

[sijie]

Original Issue: apache/pulsar#13299

---

Motivation

We decide to release 2.7.4, but we meet some flaky tests in branch-2.7. We will disable these flaky tests temporarily and use this issue to track them and if anyone is interested in fixing, feel free to push the pr.

Flaky tests:

• AdminApiTest2#testMaxSubPerTopicApi

  2021-12-14T11:03:47.4263908Z [ERROR]   Run 2: AdminApiTest2.testMaxSubPerTopicApi:1574 expected [null] but found [200]

• AdminTest#properties

  2021-12-14T11:03:47.4304265Z [ERROR]   Run 2: AdminTest.properties:521 expected [500] but found [404]

• TopicReaderTest#testMessageAvailableAfterRestart

  2021-12-14T10:24:07.9855447Z [ERROR]   Run 2: TopicReaderTest.testMessageAvailableAfterRestart:944 expected [false] but found [true]

• KeySharedSubscriptionTest#testRemoveFirstConsumer

  2021-12-14T10:24:07.9849884Z [ERROR]   Run 2: KeySharedSubscriptionTest.testRemoveFirstConsumer:690 expected [0] but found [10]

• DispatcherBlockConsumerTest#testBlockBrokerDispatching

  Error:  testBlockBrokerDispatching(org.apache.pulsar.client.api.DispatcherBlockConsumerTest)  Time elapsed: 5.58 s  <<< FAILURE!
  java.lang.AssertionError: expected [600] but found [495]
  at org.apache.pulsar.client.api.DispatcherBlockConsumerTest.testBlockBrokerDispatching(DispatcherBlockConsumerTest.java:816)

• ReplicatorTest#testReplicatorWithPartitionedTopic

  [ERROR] Tests run: 27, Failures: 1, Errors: 0, Skipped: 2, Time elapsed: 171.283 s <<< FAILURE! - in org.apache.pulsar.broker.service.ReplicatorTest
  2021-12-15T00:18:46.8530491Z [ERROR] testReplicatorWithPartitionedTopic(org.apache.pulsar.broker.service.ReplicatorTest)  Time elapsed: 30.239 s  <<< FAILURE!
  2021-12-15T00:18:46.8533510Z org.apache.pulsar.client.admin.PulsarAdminException$ServerSideErrorException: HTTP 500 Internal Server Errorjava.io.ByteArrayInputStream@a5bfb6

• MessagePublishBufferThrottleTest#testBlockByPublishRateLimiting

  Error:  testBlockByPublishRateLimiting(org.apache.pulsar.broker.service.MessagePublishBufferThrottleTest)  Time elapsed: 1.505 s  <<< FAILURE!
  java.lang.AssertionError: expected [0] but found [1048628]
  at org.apache.pulsar.broker.service.MessagePublishBufferThrottleTest.testBlockByPublishRateLimiting(MessagePublishBufferThrottleTest.java:156)

• MessagePublishBufferThrottleTest#testMessagePublishBufferThrottleEnable

  Error:  testMessagePublishBufferThrottleEnable(org.apache.pulsar.broker.service.MessagePublishBufferThrottleTest)  Time elapsed: 1.482 s  <<< FAILURE!
  java.lang.AssertionError: expected [0] but found [1048628]
  at org.apache.pulsar.broker.service.MessagePublishBufferThrottleTest.testMessagePublishBufferThrottleEnable(MessagePublishBufferThrottleTest.java:111)

• TopicReaderTest#testMultiReaderMessageAvailableAfterRestart

  TopicReaderTest.testMultiReaderMessageAvailableAfterRestart:982 expected [false] but found [true]

• TopicReaderTest#testMessageAvailableAfterRestart

  TopicReaderTest.testMessageAvailableAfterRestart:946 expected [false] but found [true]

• NonPersistentTopicE2ETest#testGCWillDeleteSchema

  NonPersistentTopicE2ETest.testGCWillDeleteSchema:106 expected [false] but found [true]

• SubscriptionSeekTest#testSeekForBatchMessageAndSpecifiedBatchIndex

  SubscriptionSeekTest.testSeekForBatchMessageAndSpecifiedBatchIndex:237 ? ConsumerBusy

• PeerReplicatorTest#testPeerClusterTopicLookup

  PeerReplicatorTest.testPeerClusterTopicLookup ? Conflict Namespace already existing or didn't finish within the time-out 10000

• PrometheusMetricsTest#testAuthMetrics

  PrometheusMetricsTest.testAuthMetrics:771 expected [token] but found [tls]

• PersistentTopicsTest#testPeekWithSubscriptionNameNotExist

  PersistentTopicsTest.testPeekWithSubscriptionNameNotExist ? ServerSideError HTTP ERROR

• PulsarStateTest#testSinkState

  java.lang.AssertionError: expected [val1-9] but found [val1-3]
  at org.apache.pulsar.tests.integration.functions.PulsarStateTest.testSinkState(PulsarStateTest.java:182)

• SmokeTest#setup

  Error:  setup(org.apache.pulsar.tests.integration.SmokeTest)  Time elapsed: 302.441 s  <<< FAILURE!
  org.testcontainers.containers.ContainerLaunchException: Container startup failed
  Caused by: org.testcontainers.containers.ContainerLaunchException: Timed out waiting for URL to be accessible (http://localhost:49184/admin/v2/namespaces/public/default should return HTTP [200])

• CurrentLedgerRolloverIfFullTest#testCurrentLedgerRolloverIfFull

  CurrentLedgerRolloverIfFullTest.testCurrentLedgerRolloverIfFull:78 ? ConditionTimeout

• ManagedLedgerCompressionTest#testRestartBrokerEnableManagedLedgerInfoCompression

  ManagedLedgerCompressionTest.testRestartBrokerEnableManagedLedgerInfoCompression ? ThreadTimeout

• PulsarStateTest

• PulsarFunctionsThreadTest

• PulsarFunctionsTest

  [ERROR] Tests run: 19, Failures: 1, Errors: 0, Skipped: 1, Time elapsed: 903.512 s <<< FAILURE! - in TestSuite
  2021-12-15T07:25:52.1420290Z [ERROR] pulsar-test-suite(org.apache.pulsar.tests.integration.functions.PulsarFunctionsProcessTest)  Time elapsed: 44.145 s  <<< FAILURE!
  2021-12-15T07:25:52.1422739Z java.lang.NullPointerException
  2021-12-15T07:25:52.1425824Z    at org.apache.pulsar.tests.integration.functions.PulsarFunctionsTest.publishAndConsumeMessages(PulsarFunctionsTest.java:2705)

• AdminApiTlsAuthTest#testDeleteNamespace

  org.apache.pulsar.broker.admin.AdminApiTlsAuthTest.testDeleteNamespace(org.apache.pulsar.broker.admin.AdminApiTlsAuthTest)
  2021-12-15T08:02:24.2151225Z [INFO]   Run 1: PASS
  2021-12-15T08:02:24.2160253Z [ERROR]   Run 2: AdminApiTlsAuthTest.testDeleteNamespace:363 ? PulsarClient java.util.concurren

• PulsarFunctionsProcessTest#publishAndConsumeMessages

  2021-12-15T09:55:34.0951669Z [ERROR] pulsar-test-suite(org.apache.pulsar.tests.integration.functions.PulsarFunctionsProcessTest)  Time elapsed: 43.382 s  <<< FAILURE!
  2021-12-15T09:55:34.0954063Z java.lang.NullPointerException
  2021-12-15T09:55:34.0957121Z    at org.apache.pulsar.tests.integration.functions.PulsarFunctionsTest.publishAndConsumeMessages(PulsarFunctionsTest.java:2705)

• NegativeAcksTest#testNegativeAcks

  NegativeAcksTest.testNegativeAcks:151 expected [null] but found [org.apache.pulsar.client.impl.MessageImpl@22b3ef5]

• PersistentDispatcherFailoverConsumerTest#testAddRemoveConsumerNonPartitionedTopic

  2021-12-15T13:07:53.3690065Z [ERROR]   Run 2: PersistentDispatcherFailoverConsumerTest.testAddRemoveConsumerNonPartitionedTopic:508
  2021-12-15T13:07:53.3692327Z Wanted but not invoked:
  2021-12-15T13:07:53.3693128Z consumer.notifyActiveConsumerChange(
  2021-12-15T13:07:53.3696542Z     same(Consumer$MockitoMock$1319454305{subscription=PersistentSubscription{topic=persistent://part-perf/global/perf.t1/ptopic, name=sub-1}, consumerId=2, consumerName=Cons2, address=localhost/127.0.0.1:1234})
  2021-12-15T13:07:53.3697923Z );
  2021-12-15T13:07:53.3702608Z -> at org.apache.pulsar.broker.service.PersistentDispatcherFailoverConsumerTest.testAddRemoveConsumerNonPartitionedTopic(PersistentDispatcherFailoverConsumerTest.java:508)
  2021-12-15T13:07:53.3706917Z
  2021-12-15T13:07:53.3707436Z However, there was exactly 1 interaction with this mock:

• PulsarOffsetBackingStoreTest#testGetFromEmptyCallback

  [ERROR] testGetFromEmptyCallback(org.apache.pulsar.io.kafka.connect.PulsarOffsetBackingStoreTest)  Time elapsed: 300.009 s  <<< FAILURE!
  2021-12-15T16:29:12.3112213Z org.testng.internal.thread.ThreadTimeoutException: Method org.apache.pulsar.io.kafka.connect.PulsarOffsetBackingStoreTest.testGetFromEmptyCallback() didn't finish within the time-out 300000

• DeadLetterTopicTest#testDeadLetterTopicByCustomTopicName

  DeadLetterTopicTest.testDeadLetterTopicByCustomTopicName:397 expected [null] but found [org.apache.pulsar.client.impl.MessageImpl@2c2e4458]

• TopicReaderTest#testMultiReaderBuilderConcurrentCreate

  TopicReaderTest.testMultiReaderBuilderConcurrentCreate ? Conflict This topic already exsiting

[github-actions[bot]]

The issue had no activity for 30 days, mark with Stale label.