redpanda-data / redpanda

Redpanda is a streaming data platform for developers. Kafka API compatible. 10x faster. No ZooKeeper. No JVM!
https://redpanda.com
9.36k stars 575 forks source link

CI Failure (key symptom) in `TopicDeleteTest.topic_delete_test` #17074

Open vbotbuildovich opened 5 months ago

vbotbuildovich commented 5 months ago

https://buildkite.com/redpanda/vtools/builds/12235#018e3442-7521-4176-8dd4-b7cf61d4dff9 https://buildkite.com/redpanda/vtools/builds/12235#018e3442-7521-4176-8dd4-b7cf61d4dff9 https://buildkite.com/redpanda/vtools/builds/12243#018e36d6-b2a8-4e22-acba-eaee1726dd61 https://buildkite.com/redpanda/vtools/builds/12243#018e36d6-b2a8-4e22-acba-eaee1726dd61

Module: rptest.tests.topic_delete_test
Class: TopicDeleteTest
Method: topic_delete_test
Arguments: {
    "with_restart": false
}
test_id:    TopicDeleteTest.topic_delete_test
status:     FAIL
run time:   13.130 seconds

RemoteCommandError({'ssh_config': {'host': 'ip-172-31-41-56', 'hostname': '172.31.41.56', 'user': 'root', 'port': 22, 'password': None, 'identityfile': '/home/ubuntu/.ssh/id_rsa'}, 'hostname': 'ip-172-31-41-56', 'ssh_hostname': '172.31.41.56', 'user': 'root', 'externally_routable_ip': '35.92.203.82', '_logger': <Logger rptest.tests.topic_delete_test.TopicDeleteTest.topic_delete_test.with_restart=False-1314 (DEBUG)>, 'os': 'linux', '_ssh_client': <paramiko.client.SSHClient object at 0xffff9084c9a0>, '_sftp_client': <paramiko.sftp_client.SFTPClient object at 0xffff9080d420>, '_custom_ssh_exception_checks': None}, 'python3 /opt/scripts/offline_log_viewer/viewer.py --path /var/lib/redpanda/data --type kvstore', 1, b'Traceback (most recent call last):\n  File "/opt/scripts/offline_log_viewer/viewer.py", line 6, in <module>\n    from controller import ControllerLog\n  File "/opt/scripts/offline_log_viewer/controller.py", line 5, in <module>\n    from storage import Segment\n  File "/opt/scripts/offline_log_viewer/storage.py", line 8, in <module>\n    import crc32c\nModuleNotFoundError: No module named \'crc32c\'\n')
Traceback (most recent call last):
  File "/home/ubuntu/.ducktape-venv/lib/python3.10/site-packages/ducktape/tests/runner_client.py", line 184, in _do_run
    data = self.run_test()
  File "/home/ubuntu/.ducktape-venv/lib/python3.10/site-packages/ducktape/tests/runner_client.py", line 269, in run_test
    return self.test_context.function(self.test)
  File "/home/ubuntu/.ducktape-venv/lib/python3.10/site-packages/ducktape/mark/_mark.py", line 481, in wrapper
    return functools.partial(f, *args, **kwargs)(*w_args, **w_kwargs)
  File "/home/ubuntu/redpanda/tests/rptest/services/cluster.py", line 104, in wrapped
    r = f(self, *args, **kwargs)
  File "/home/ubuntu/redpanda/tests/rptest/tests/topic_delete_test.py", line 176, in topic_delete_test
    assert sum(get_kvstore_topic_key_counts(
  File "/home/ubuntu/redpanda/tests/rptest/tests/topic_delete_test.py", line 69, in get_kvstore_topic_key_counts
    kvstore_data = viewer.read_kvstore(node=n)
  File "/home/ubuntu/redpanda/tests/rptest/clients/offline_log_viewer.py", line 27, in read_kvstore
    kvstore_json = node.account.ssh_output(cmd, combine_stderr=False)
  File "/home/ubuntu/.ducktape-venv/lib/python3.10/site-packages/ducktape/cluster/remoteaccount.py", line 41, in wrapper
    return method(self, *args, **kwargs)
  File "/home/ubuntu/.ducktape-venv/lib/python3.10/site-packages/ducktape/cluster/remoteaccount.py", line 397, in ssh_output
    raise RemoteCommandError(self, cmd, exit_status, stderr.read())
ducktape.cluster.remoteaccount.RemoteCommandError: root@ip-172-31-41-56: Command 'python3 /opt/scripts/offline_log_viewer/viewer.py --path /var/lib/redpanda/data --type kvstore' returned non-zero exit status 1. Remote error message: b'Traceback (most recent call last):\n  File "/opt/scripts/offline_log_viewer/viewer.py", line 6, in <module>\n    from controller import ControllerLog\n  File "/opt/scripts/offline_log_viewer/controller.py", line 5, in <module>\n    from storage import Segment\n  File "/opt/scripts/offline_log_viewer/storage.py", line 8, in <module>\n    import crc32c\nModuleNotFoundError: No module named \'crc32c\'\n'

JIRA Link: CORE-5634

travisdowns commented 5 months ago

Duplicate of https://github.com/redpanda-data/redpanda/issues/17042.

travisdowns commented 5 months ago

Duplicate of #17042

travisdowns commented 5 months ago

Duplicate of https://github.com/redpanda-data/redpanda/issues/17042

vbotbuildovich commented 1 month ago

https://buildkite.com/redpanda/vtools/builds/12268 https://buildkite.com/redpanda/vtools/builds/12268 *https://buildkite.com/redpanda/redpanda/builds/51499