stackabletech / hdfs-operator

Apache Hadoop HDFS operator for Stackable
Other
40 stars 4 forks source link

Investigate test failures related to WebHDFS and ListenerClass external-unstable #467

Closed sbernauer closed 9 months ago

sbernauer commented 9 months ago

We are running into actual test failures related to WebHdfs in case external-unstable is used: logger.go:42: 09:05:06 | smoke_hadoop-3.3.4_zookeeper-3.8.3_zookeeper-latest-3.8.3_number-of-datanodes-2_datanode-pvcs-2hdd-1ssd_listener-class-external-unstable/50-create-file | requests.exceptions.ConnectionError: HTTPConnectionPool(host='hdfs-datanode-default-0.hdfs-datanode-default.kuttl-test-exciting-sloth.svc.cluster.local', port=32626): Max retries exceeded with url: /webhdfs/v1/testdata.txt?op=CREATE&user.name=stackable&namenoderpcaddress=hdfs&createflag=&createparent=true&overwrite=false (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fbe5b054a50>: Failed to establish a new connection: [Errno 111] Connection refused'))

This is probably caused because the POST to the NameNode redirects to a wrong DataNode address in the tests https://github.com/stackabletech/hdfs-operator/blob/cf39abe1763d1e3bc794c5dfbc664e73b3ab6c53/tests/templates/kuttl/smoke/webhdfs.py#L34