This PR contains multiple fixes for the hadoop-on-local-fs scenario, i.e., the one where fs.defaultFS is set to file:///, and adds a new Travis job to test it.
Main highlights:
Hadoop's pipes submitter is still broken due to MAPREDUCE-4000. However, this PR applies the required fix to our own submitter, so Pydoop now supports running mapreduce applications on the local fs via pydoop submit.
The above issue further stresses the fact that the "official" way to run applications is via pydoop submit. Examples still relying on PipesRunner have been rewritten to use pydoop submit, so that they can run correctly on the local fs. Running via mapred pipes is still supported when it works, and we're checking that in int_test/mapred_submitter (which serves as a sanity check for our submitter).
PipesRunner and related stuff can now be dropped, since they would just be a manteinance burden.
This PR contains multiple fixes for the hadoop-on-local-fs scenario, i.e., the one where
fs.defaultFSis set tofile:///, and adds a new Travis job to test it.Main highlights:
pydoop submit.pydoop submit. Examples still relying onPipesRunnerhave been rewritten to usepydoop submit, so that they can run correctly on the local fs. Running viamapred pipesis still supported when it works, and we're checking that inint_test/mapred_submitter(which serves as a sanity check for our submitter).PipesRunnerand related stuff can now be dropped, since they would just be a manteinance burden.