MapReduce & Hadoop

[HorizonNet]

Below is a short overview.

Tasks

• [x] Download a book text from the Gutenberg Project and copy it into HDFS
• [x] Write a MapReduce program that does a WordCount on an arbitrary text file
• [x] Package your WordCount application into a Maven project and build it
• [x] Using teragen and terasort

Below are a few comments/suggestions:

• Use hdfs dfs instead of hadoop fs (see this link).
• The version of hadoop-core in your Maven application is very old. Use a recent version instead (f.e. 2.7.3).
• If you commit changes from within the Quickstart VM a different Git user name is used. This should be reconfigured. See this commit for details.
• Next time it would be great if you could also add the command output, because it contains some useful information (see your hdfs-tests.md file).

Summary:

You're done with this one. Good work.

[achintya-kumar]

Thank you! I'll make the changes you have suggested.