redpanda-data / redpanda

Redpanda is a streaming data platform for developers. Kafka API compatible. 10x faster. No ZooKeeper. No JVM!
https://redpanda.com
9.16k stars 561 forks source link

CI Failure (Timeout in producer.wait_for_acks) in `IdempotencyStressTest.producer_id_stress_test` #18273

Open vbotbuildovich opened 2 months ago

vbotbuildovich commented 2 months ago

https://buildkite.com/redpanda/vtools/builds/13501

Module: rptest.tests.idempotency_stress_test
Class: IdempotencyStressTest
Method: producer_id_stress_test
Arguments: {
    "max_producer_ids": 100
}
test_id:    IdempotencyStressTest.producer_id_stress_test
status:     FAIL
run time:   837.874 seconds

TimeoutError('')
Traceback (most recent call last):
  File "/opt/.ducktape-venv/lib/python3.10/site-packages/ducktape/tests/runner_client.py", line 184, in _do_run
    data = self.run_test()
  File "/opt/.ducktape-venv/lib/python3.10/site-packages/ducktape/tests/runner_client.py", line 276, in run_test
    return self.test_context.function(self.test)
  File "/opt/.ducktape-venv/lib/python3.10/site-packages/ducktape/mark/_mark.py", line 535, in wrapper
    return functools.partial(f, *args, **kwargs)(*w_args, **w_kwargs)
  File "/home/ubuntu/redpanda/tests/rptest/services/cluster.py", line 103, in wrapped
    r = f(self, *args, **kwargs)
  File "/home/ubuntu/redpanda/tests/rptest/tests/idempotency_stress_test.py", line 116, in producer_id_stress_test
    producer.wait_for_acks(self.msg_cnt, 600, 1)
  File "/home/ubuntu/redpanda/tests/rptest/services/kgo_verifier_services.py", line 607, in wait_for_acks
    self._redpanda.wait_until(
  File "/home/ubuntu/redpanda/tests/rptest/services/redpanda.py", line 963, in wait_until
    wait_until(wrapped,
  File "/opt/.ducktape-venv/lib/python3.10/site-packages/ducktape/utils/util.py", line 57, in wait_until
    raise TimeoutError(err_msg() if callable(err_msg) else err_msg) from last_exception
ducktape.errors.TimeoutError

JIRA Link: CORE-2805

travisdowns commented 2 weeks ago

Similar prior issue: https://github.com/redpanda-data/redpanda/issues/14178