CI Failure (key symptom) in `RandomNodeOperationsTest.test_node_operations`

[vbotbuildovich]

https://buildkite.com/redpanda/vtools/builds/15577

Module: rptest.tests.random_node_operations_test
Class: RandomNodeOperationsTest
Method: test_node_operations
Arguments: {
    "num_to_upgrade": 3,
    "enable_failures": true,
    "with_tiered_storage": true
}

test_id:    RandomNodeOperationsTest.test_node_operations
status:     FAIL
run time:   3652.591 seconds

<BadLogLines nodes=ip-172-31-11-150(1) example="ERROR 2024-07-12 23:22:55,778 [shard 2:au  ] archival - [fiber338 kafka/tp-workload-writecaching-compact/3] - ntp_archiver_service.cc:1229 - failed to upload segment "266d21db/kafka/tp-workload-writecaching-compact/3_71/25146-36954-13114178-2-v1.log.5": seastar::nested_exception (seastar::nested_exception)">
Traceback (most recent call last):
  File "/opt/.ducktape-venv/lib/python3.10/site-packages/ducktape/tests/runner_client.py", line 184, in _do_run
    data = self.run_test()
  File "/opt/.ducktape-venv/lib/python3.10/site-packages/ducktape/tests/runner_client.py", line 276, in run_test
    return self.test_context.function(self.test)
  File "/opt/.ducktape-venv/lib/python3.10/site-packages/ducktape/mark/_mark.py", line 535, in wrapper
    return functools.partial(f, *args, **kwargs)(*w_args, **w_kwargs)
  File "/home/ubuntu/redpanda/tests/rptest/services/cluster.py", line 190, in wrapped
    redpanda.raise_on_bad_logs(
  File "/home/ubuntu/redpanda/tests/rptest/services/redpanda.py", line 1576, in raise_on_bad_logs
    lsearcher.search_logs(_searchable_nodes)
  File "/home/ubuntu/redpanda/tests/rptest/services/utils.py", line 197, in search_logs
    raise BadLogLines(bad_loglines)
rptest.services.utils.BadLogLines: <BadLogLines nodes=ip-172-31-11-150(1) example="ERROR 2024-07-12 23:22:55,778 [shard 2:au  ] archival - [fiber338 kafka/tp-workload-writecaching-compact/3] - ntp_archiver_service.cc:1229 - failed to upload segment "266d21db/kafka/tp-workload-writecaching-compact/3_71/25146-36954-13114178-2-v1.log.5": seastar::nested_exception (seastar::nested_exception)">

JIRA Link: CORE-5606

[vbotbuildovich]

*https://buildkite.com/redpanda/vtools/builds/15879

[vbotbuildovich]

*https://buildkite.com/redpanda/vtools/builds/15879

[vbotbuildovich]

*https://buildkite.com/redpanda/vtools/builds/15925

[vbotbuildovich]

*https://buildkite.com/redpanda/vtools/builds/15934

[vbotbuildovich]

https://buildkite.com/redpanda/vtools/builds/15964 https://buildkite.com/redpanda/vtools/builds/15979 https://buildkite.com/redpanda/vtools/builds/15978 https://buildkite.com/redpanda/vtools/builds/15978

[vbotbuildovich]

https://buildkite.com/redpanda/vtools/builds/15964 https://buildkite.com/redpanda/vtools/builds/15978