search

redpanda-data / redpanda

Redpanda is a streaming data platform for developers. Kafka API compatible. 10x faster. No ZooKeeper. No JVM!
https://redpanda.com
9.37k stars 577 forks source link

CI Failure (key symptom) in `EndToEndHydrationTimeoutTest.test_hydration_completes_on_timeout` #20196

Closed vbotbuildovich closed 2 months ago

vbotbuildovich commented 2 months ago

https://buildkite.com/redpanda/vtools/builds/15023

Module: rptest.tests.e2e_shadow_indexing_test
Class: EndToEndHydrationTimeoutTest
Method: test_hydration_completes_on_timeout
test_id:    EndToEndHydrationTimeoutTest.test_hydration_completes_on_timeout
status:     FAIL
run time:   219.015 seconds

TimeoutError('Segments were not removed from all nodes')
Traceback (most recent call last):
  File "/opt/.ducktape-venv/lib/python3.10/site-packages/ducktape/tests/runner_client.py", line 184, in _do_run
    data = self.run_test()
  File "/opt/.ducktape-venv/lib/python3.10/site-packages/ducktape/tests/runner_client.py", line 276, in run_test
    return self.test_context.function(self.test)
  File "/home/ubuntu/redpanda/tests/rptest/services/cluster.py", line 105, in wrapped
    r = f(self, *args, **kwargs)
  File "/home/ubuntu/redpanda/tests/rptest/tests/e2e_shadow_indexing_test.py", line 1490, in test_hydration_completes_on_timeout
    self.workload()
  File "/home/ubuntu/redpanda/tests/rptest/tests/e2e_shadow_indexing_test.py", line 1449, in workload
    wait_for_removal_of_n_segments(redpanda=self.redpanda,
  File "/home/ubuntu/redpanda/tests/rptest/util.py", line 211, in wait_for_removal_of_n_segments
    wait_until(segments_removed,
  File "/opt/.ducktape-venv/lib/python3.10/site-packages/ducktape/utils/util.py", line 57, in wait_until
    raise TimeoutError(err_msg() if callable(err_msg) else err_msg) from last_exception
ducktape.errors.TimeoutError: Segments were not removed from all nodes

JIRA Link: CORE-4543

michael-redpanda commented 2 months ago

Fixed by: https://github.com/redpanda-data/redpanda/pull/20310