eecs485staff / madoop

A light weight MapReduce framework for education
MIT License
9 stars 4 forks source link

Support single input file #67

Closed awdeorio closed 8 months ago

awdeorio commented 8 months ago

Add support for a single input file.

Validation

$ tree crawl/
crawl/
├── 00001.html
├── 00002.html
├── 00003.html
└── MANIFEST.txt

1 directory, 4 files

$ cat crawl/MANIFEST.txt 
00001.html
00002.html
00003.html

$ rm -rf output
madoop \
  -input crawl/MANIFEST.txt \
  -output output \
  -mapper /bin/cat \
  -reducer /bin/cat
cat output/*
INFO: Starting map stage
INFO: Finished map executions: 1
INFO: Starting group stage
INFO: Starting reduce stage
INFO: Finished reduce executions: 3
INFO: Output directory: output
00001.html
00002.html
00003.html
codecov[bot] commented 8 months ago

Codecov Report

All modified and coverable lines are covered by tests :white_check_mark:

Files Coverage Δ
madoop/__main__.py 93.75% <ø> (ø)
madoop/mapreduce.py 96.96% <100.00%> (+0.19%) :arrow_up:

:loudspeaker: Thoughts on this report? Let us know!.

awdeorio commented 8 months ago

LGTM thanks for the addition