erikfrey / bashreduce

mapreduce in bash
MIT License
920 stars 88 forks source link

Question: How to use 'poor man's dfs'? #9

Open dakusui opened 9 years ago

dakusui commented 9 years ago

In README, it is said that we can use bashreduce as a poorman's dfs by doing

br -r "cat > /tmp/myfile" < input

I think I can see a file 'input' will be distributed among the hosts with the file name /tmp/myfile. But how can we read the file?

br -r "cat /tmp/myfile"

would give us a broken file since those distributed chunks do not have information about in what order the original file was written. Or it is assumed that input is a file whose lines have numbers in order?