Kimahriman / hdfs-native

Apache License 2.0
33 stars 12 forks source link

Support DataNode connection reuse #80

Closed Kimahriman closed 9 months ago

Kimahriman commented 9 months ago

Resolves #79

Adds a static DataNode connection cache with a three second expiry, the default the Hadoop client uses. Connections are only purged from the cache when a connection is retrieved from the cache, and we will try up to three cached connections before falling back to creating a new connection. This should help speed up multiple reads to the same file/DataNode in fast succession.