CI Failure (anomalies changed after restart) in `CloudStorageScrubberTest.test_scrubber`

[rockwotj]

https://buildkite.com/redpanda/redpanda/builds/39080

Module: rptest.tests.cloud_storage_scrubber_test
Class: CloudStorageScrubberTest
Method: test_scrubber
Arguments: {
    "cloud_storage_type": 1
}

test_id:    CloudStorageScrubberTest.test_scrubber
status:     FAIL
run time:   161.655 seconds

TimeoutError('Reported anomalies changed after full restart')
Traceback (most recent call last):
  File "/usr/local/lib/python3.10/dist-packages/ducktape/tests/runner_client.py", line 184, in _do_run
    data = self.run_test()
  File "/usr/local/lib/python3.10/dist-packages/ducktape/tests/runner_client.py", line 269, in run_test
    return self.test_context.function(self.test)
  File "/usr/local/lib/python3.10/dist-packages/ducktape/mark/_mark.py", line 481, in wrapper
    return functools.partial(f, *args, **kwargs)(*w_args, **w_kwargs)
  File "/root/tests/rptest/services/cluster.py", line 82, in wrapped
    r = f(self, *args, **kwargs)
  File "/root/tests/rptest/tests/cloud_storage_scrubber_test.py", line 354, in test_scrubber
    self._assert_anomalies_stable_after_restart()
  File "/root/tests/rptest/tests/cloud_storage_scrubber_test.py", line 267, in _assert_anomalies_stable_after_restart
    wait_until(anomalies_stable,
  File "/usr/local/lib/python3.10/dist-packages/ducktape/utils/util.py", line 57, in wait_until
    raise TimeoutError(err_msg() if callable(err_msg) else err_msg) from last_exception
ducktape.errors.TimeoutError: Reported anomalies changed after full restart

JIRA Link: CORE-1513

[VladLazar]

I think I broke this in https://github.com/redpanda-data/redpanda/pull/13972, but it should be fixed with https://github.com/redpanda-data/redpanda/pull/14196.

[VladLazar]

Should be fixed now via https://github.com/redpanda-data/redpanda/pull/14196

[abhijat]

reoccured in https://buildkite.com/redpanda/redpanda/builds/41388#018be65e-3a33-4cd3-a1c3-bc5b753c52ea

[dotnwat]

All of the logs have been GCd since the last report :(