CI Failure (key symptom) in `DataMigrationsApiTest.test_creating_and_listing_migrations`

[vbotbuildovich]

https://buildkite.com/redpanda/redpanda/builds/51167

Module: rptest.tests.data_migrations_api_test
Class: DataMigrationsApiTest
Method: test_creating_and_listing_migrations

test_id:    DataMigrationsApiTest.test_creating_and_listing_migrations
status:     FAIL
run time:   60.886 seconds

ConnectionError(MaxRetryError("HTTPConnectionPool(host='docker-rp-2', port=9644): Max retries exceeded with url: /v1/migrations (Caused by ProtocolError('Connection aborted.', ConnectionResetError(104, 'Connection reset by peer')))"))
Traceback (most recent call last):
  File "/opt/.ducktape-venv/lib/python3.10/site-packages/urllib3/connectionpool.py", line 670, in urlopen
    httplib_response = self._make_request(
  File "/opt/.ducktape-venv/lib/python3.10/site-packages/urllib3/connectionpool.py", line 426, in _make_request
    six.raise_from(e, None)
  File "<string>", line 3, in raise_from
  File "/opt/.ducktape-venv/lib/python3.10/site-packages/urllib3/connectionpool.py", line 421, in _make_request
    httplib_response = conn.getresponse()
  File "/usr/lib/python3.10/http/client.py", line 1375, in getresponse
    response.begin()
  File "/usr/lib/python3.10/http/client.py", line 318, in begin
    version, status, reason = self._read_status()
  File "/usr/lib/python3.10/http/client.py", line 279, in _read_status
    line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
  File "/usr/lib/python3.10/socket.py", line 705, in readinto
    return self._sock.recv_into(b)
ConnectionResetError: [Errno 104] Connection reset by peer

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/opt/.ducktape-venv/lib/python3.10/site-packages/requests/adapters.py", line 439, in send
    resp = conn.urlopen(
  File "/opt/.ducktape-venv/lib/python3.10/site-packages/urllib3/connectionpool.py", line 726, in urlopen
    retries = retries.increment(
  File "/opt/.ducktape-venv/lib/python3.10/site-packages/urllib3/util/retry.py", line 446, in increment
    raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='docker-rp-2', port=9644): Max retries exceeded with url: /v1/migrations (Caused by ProtocolError('Connection aborted.', ConnectionResetError(104, 'Connection reset by peer')))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/opt/.ducktape-venv/lib/python3.10/site-packages/ducktape/tests/runner_client.py", line 184, in _do_run
    data = self.run_test()
  File "/opt/.ducktape-venv/lib/python3.10/site-packages/ducktape/tests/runner_client.py", line 276, in run_test
    return self.test_context.function(self.test)
  File "/root/tests/rptest/services/cluster.py", line 105, in wrapped
    r = f(self, *args, **kwargs)
  File "/root/tests/rptest/tests/data_migrations_api_test.py", line 107, in test_creating_and_listing_migrations
    self.wait_for_migration_state(out_migration_id, 'prepared')
  File "/root/tests/rptest/tests/data_migrations_api_test.py", line 38, in wait_for_migration_state
    wait_until(
  File "/opt/.ducktape-venv/lib/python3.10/site-packages/ducktape/utils/util.py", line 53, in wait_until
    raise e
  File "/opt/.ducktape-venv/lib/python3.10/site-packages/ducktape/utils/util.py", line 44, in wait_until
    if condition():
  File "/root/tests/rptest/tests/data_migrations_api_test.py", line 33, in migration_in_state
    migrations = self.get_migrations_map(n)
  File "/root/tests/rptest/tests/data_migrations_api_test.py", line 27, in get_migrations_map
    migrations = self.admin.list_data_migrations(node).json()
  File "/root/tests/rptest/services/admin.py", line 1671, in list_data_migrations
    return self._request("GET", path, node=node)
  File "/root/tests/rptest/services/admin.py", line 640, in _request
    r = self._session.request(verb, url, **kwargs)
  File "/opt/.ducktape-venv/lib/python3.10/site-packages/requests/sessions.py", line 530, in request
    resp = self.send(prep, **send_kwargs)
  File "/opt/.ducktape-venv/lib/python3.10/site-packages/requests/sessions.py", line 643, in send
    r = adapter.send(request, **kwargs)
  File "/opt/.ducktape-venv/lib/python3.10/site-packages/requests/adapters.py", line 516, in send
    raise ConnectionError(e, request=request)
requests.exceptions.ConnectionError: HTTPConnectionPool(host='docker-rp-2', port=9644): Max retries exceeded with url: /v1/migrations (Caused by ProtocolError('Connection aborted.', ConnectionResetError(104, 'Connection reset by peer')))

JIRA Link: CORE-5524

[vbotbuildovich]

*https://buildkite.com/redpanda/redpanda/builds/51183

[vbotbuildovich]

*https://buildkite.com/redpanda/redpanda/builds/51232

[vbotbuildovich]

*https://buildkite.com/redpanda/redpanda/builds/51672