search

ClickHouse / libhdfs3

HDFS file read access for ClickHouse
Apache License 2.0
36 stars 56 forks source link

optimize hdfs seek #31

Closed M1eyu2018 closed 1 year ago

M1eyu2018 commented 2 years ago

After optimizing the hdfs seek method, if the target of the seek method exceeds a certain length, the blockReader's skip method will not be called again to prevent frequent calls to the readNextPacket method, thus improving the performance of the seek method.

CLAassistant commented 2 years ago

CLA assistant check
All committers have signed the CLA.

zhanglistar commented 1 year ago

@alexey-milovidov @alesapin @kssenii This code is same as Java client, and we have tested in our env. which will reduce network flow when seeking offset is too long from current offset result a performance improvement. Please merge. Thanks.

zhanglistar commented 1 year ago

@alexey-milovidov Do you want to merge this?

alexey-milovidov commented 1 year ago

Thank you! We missed this PR.