ClickHouse / libhdfs3

HDFS file read access for ClickHouse
Apache License 2.0
36 stars 56 forks source link

Fix hang erasure encoding data in HDFS and another assertion bug #20

Closed zhanglistar closed 2 years ago

zhanglistar commented 2 years ago

Two changes: 1) Now libhdfs3 does not support erasure encoding data in HDFS, causing clickhouse hangs. We just throw "not supported" hint for now, and we will develop libhdfs3 to support EC in later days.

image

2) Fix another assertion fail in debug mode. According this https://github.com/apache/hadoop/blob/62c86eaa0e539a4307ca794e0fcd502a77ebceb8/hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocolPB/PBHelperClient.java#L641 https://github.com/apache/hadoop/blob/62c86eaa0e539a4307ca794e0fcd502a77ebceb8/hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocolPB/PBHelperClient.java#L800 stack: image

zhanglistar commented 2 years ago

@alesapin Please check.