linkedin / dynamometer

A tool for scale and performance testing of HDFS with a specific focus on the NameNode.
BSD 2-Clause "Simplified" License
131 stars 34 forks source link

Fix usage of DFSClient (follow-on to #1) #7

Closed xkrogen closed 6 years ago

xkrogen commented 6 years ago

1 used direct access to DFSClient to perform more accurate listing operations. To access the DFSClient from within DistributedFileSystem, a utility was added in the o.a.h.hdfs package to access the package-private dfs field. In the hadoopRuntime (default) configuration, the o.a.h package is excluded, so though this works fine in the bundled integration test, it fails when run from the generated zip.

I noticed that DistributedFileSystem exports a public getClient() method which we can use instead. It's marked @VisibleForTesting, but is still less hacky than using a workaround to access package-private field.