redpanda-data / redpanda

Redpanda is a streaming data platform for developers. Kafka API compatible. 10x faster. No ZooKeeper. No JVM!
https://redpanda.com
9.39k stars 577 forks source link

CI Failure (Failed to make reupload candidate) in `RandomNodeOperationsTest.test_node_operations` #19914

Open vbotbuildovich opened 2 months ago

vbotbuildovich commented 2 months ago

https://buildkite.com/redpanda/redpanda/builds/50329

Module: rptest.tests.random_node_operations_test
Class: RandomNodeOperationsTest
Method: test_node_operations
Arguments: {
    "num_to_upgrade": 0,
    "enable_failures": false,
    "with_tiered_storage": true
}
test_id:    RandomNodeOperationsTest.test_node_operations
status:     FAIL
run time:   495.605 seconds

<BadLogLines nodes=docker-rp-8(1) example="ERROR 2024-06-16 06:35:42,710 [shard 1:au  ] archival - [fiber252 kafka/fuzzy-operator-3282-yvwkcs/0] - ntp_archiver_service.cc:2861 - Failed to make reupload candidate to match the run, candidate: {source segment offsets: {term:1, base_offset:0, committed_offset:212, dirty_offset:212}, exposed_name: 0-1-v1.log, starting_offset: 0, file_offset: 0, content_length: 2987, final_offset: 212, final_file_offset: 2987, term: 1, source names: {/var/lib/redpanda/data/kafka/fuzzy-operator-3282-yvwkcs/0_1426/0-1-v1.log}}, run: {is_compacted: false, size_bytes: 13752, base_offset: 0, committed_offset: 212, base_timestamp: {timestamp: 1718519665503}, max_timestamp: {timestamp: 1718519665504}, delta_offset: 0, ntp_revision: 65, archiver_term: 1, segment_term: 1, delta_offset_end: 7, sname_format: {v3}, metadata_size_hint: 0}">
Traceback (most recent call last):
  File "/opt/.ducktape-venv/lib/python3.10/site-packages/ducktape/tests/runner_client.py", line 184, in _do_run
    data = self.run_test()
  File "/opt/.ducktape-venv/lib/python3.10/site-packages/ducktape/tests/runner_client.py", line 276, in run_test
    return self.test_context.function(self.test)
  File "/opt/.ducktape-venv/lib/python3.10/site-packages/ducktape/mark/_mark.py", line 535, in wrapper
    return functools.partial(f, *args, **kwargs)(*w_args, **w_kwargs)
  File "/root/tests/rptest/services/cluster.py", line 190, in wrapped
    redpanda.raise_on_bad_logs(
  File "/root/tests/rptest/services/redpanda.py", line 1565, in raise_on_bad_logs
    lsearcher.search_logs(_searchable_nodes)
  File "/root/tests/rptest/services/utils.py", line 197, in search_logs
    raise BadLogLines(bad_loglines)
rptest.services.utils.BadLogLines: <BadLogLines nodes=docker-rp-8(1) example="ERROR 2024-06-16 06:35:42,710 [shard 1:au  ] archival - [fiber252 kafka/fuzzy-operator-3282-yvwkcs/0] - ntp_archiver_service.cc:2861 - Failed to make reupload candidate to match the run, candidate: {source segment offsets: {term:1, base_offset:0, committed_offset:212, dirty_offset:212}, exposed_name: 0-1-v1.log, starting_offset: 0, file_offset: 0, content_length: 2987, final_offset: 212, final_file_offset: 2987, term: 1, source names: {/var/lib/redpanda/data/kafka/fuzzy-operator-3282-yvwkcs/0_1426/0-1-v1.log}}, run: {is_compacted: false, size_bytes: 13752, base_offset: 0, committed_offset: 212, base_timestamp: {timestamp: 1718519665503}, max_timestamp: {timestamp: 1718519665504}, delta_offset: 0, ntp_revision: 65, archiver_term: 1, segment_term: 1, delta_offset_end: 7, sname_format: {v3}, metadata_size_hint: 0}">

JIRA Link: CORE-4242

vbotbuildovich commented 2 months ago

*https://buildkite.com/redpanda/vtools/builds/15021