paypal / NNAnalytics

NameNodeAnalytics is a self-help utility for scouting and maintaining the namespace of an HDFS instance.
Apache License 2.0
110 stars 71 forks source link
fsimage hadoop hdfs metadata namespace scanner utility

License Build Status Documentation Status codecov Codacy Badge Join the chat at https://gitter.im/NNAnalytics/Lobby

"A Standby read-only HDFS NameNode, with no RPC server, that services clients over a REST API, utilizes Java 8 Stream API, all for the purpose of performing large and complicated scans of the entire file system metadata for end users."

Run a demo locally and instantly! Just run the following command on a workspace directory:

git clone https://github.com/paypal/NNAnalytics.git nna && cd ./nna && ./gradlew -PmainClass=org.apache.hadoop.hdfs.server.namenode.analytics.TestWithMiniClusterWithStreamEngine execute

Then go to http://localhost:4567 and you will have an NNA instance complete with a mini HA-enabled HDFS instance all updating in real time on your local machine!


Architecture: Legacy vs NNA


Documentation & Getting Started


Presentations