MapReduce and Hadoop

[likithkailas]

Uploaded hdfs.sh shell script file under master, since the .sh file upload is not supported under issues

[likithkailas]

MapReducetasks.zip

Contains the .jar file of the map reduce task compiled and built as maven project on eclipse

[likithkailas]

HDFS_Shell_Script .zip

committing the shell script for: 1.staring the hadoop server

2. Local file path from local disk as input from user
3. Moving the file to HDFS 4.stopping the hadoop server

[likithkailas]

hdfs-tests.zip

Performed teragen and terasort ,created a 5GB file in HDFS home directoy, please find the commands and the time duration information in the file below

[HorizonNet]

Below is a short review.

Tasks

• [x] Download a book text from the Gutenberg Project and copy it into HDFS
• [x] Write a MapReduce program that does a WordCount on an arbitrary text file
• [ ] Package your WordCount application into a Maven project and build it
• [ ] Using teragen and terasort

Below are a few comments/suggestions:

• If you're working with HDFS as the underlying filesystem use hdfs dfs instead of hadoop fs. Primarily, this is up to personal preference.
• Use appropriate formatting for your Java files. This makes it easier to others to read it.
• There are other ways for splitting lines. Currently you're using a hard-coded "," as the line separator.

Open points:

• Instead of putting your results into the issue, put it into your repository. That's the whole point of using Git. Then you also don't need to put the files into an archive.