redpanda-data / redpanda

Redpanda is a streaming data platform for developers. Kafka API compatible. 10x faster. No ZooKeeper. No JVM!
https://redpanda.com
9.46k stars 579 forks source link

CI Failure (key symptom) in `RandomNodeOperationsTest.test_node_operations` #19827

Closed vbotbuildovich closed 3 months ago

vbotbuildovich commented 3 months ago

https://buildkite.com/redpanda/redpanda/builds/50143

Module: rptest.tests.random_node_operations_test
Class: RandomNodeOperationsTest
Method: test_node_operations
Arguments: {
    "num_to_upgrade": 0,
    "enable_failures": false,
    "with_tiered_storage": false
}
test_id:    RandomNodeOperationsTest.test_node_operations
status:     FAIL
run time:   398.909 seconds

<BadLogLines nodes=docker-rp-1(2) example="ERROR 2024-06-12 06:42:59,147 [shard 0:clus] assert - Assert failure: (/var/lib/buildkite-agent/builds/buildkite-amd64-builders-i-0869cf08c98d9ccf8-1/redpanda/redpanda/src/v/raft/mux_state_machine.h:344) 'it != _results.end()' last applied offset 1679 is greater than returned replicate result 1679. this must imply existence of a result in results map">
Traceback (most recent call last):
  File "/opt/.ducktape-venv/lib/python3.10/site-packages/ducktape/tests/runner_client.py", line 184, in _do_run
    data = self.run_test()
  File "/opt/.ducktape-venv/lib/python3.10/site-packages/ducktape/tests/runner_client.py", line 276, in run_test
    return self.test_context.function(self.test)
  File "/opt/.ducktape-venv/lib/python3.10/site-packages/ducktape/mark/_mark.py", line 535, in wrapper
    return functools.partial(f, *args, **kwargs)(*w_args, **w_kwargs)
  File "/root/tests/rptest/services/cluster.py", line 190, in wrapped
    redpanda.raise_on_bad_logs(
  File "/root/tests/rptest/services/redpanda.py", line 1565, in raise_on_bad_logs
    lsearcher.search_logs(_searchable_nodes)
  File "/root/tests/rptest/services/utils.py", line 197, in search_logs
    raise BadLogLines(bad_loglines)
rptest.services.utils.BadLogLines: <BadLogLines nodes=docker-rp-1(2) example="ERROR 2024-06-12 06:42:59,147 [shard 0:clus] assert - Assert failure: (/var/lib/buildkite-agent/builds/buildkite-amd64-builders-i-0869cf08c98d9ccf8-1/redpanda/redpanda/src/v/raft/mux_state_machine.h:344) 'it != _results.end()' last applied offset 1679 is greater than returned replicate result 1679. this must imply existence of a result in results map">

JIRA Link: CORE-4186

ztlpn commented 3 months ago

Duplicate of https://github.com/redpanda-data/redpanda/issues/19632