search

codeminders / hamake

Hadoop dataflow-based task manager
4 stars 1 forks source link

dist cache support #32

Closed GoogleCodeExporter closed 9 years ago

GoogleCodeExporter commented 9 years ago 
Use distributed cache for JARs with libraries used by task.

Original issue reported on code.google.com by kroko...@gmail.com on 27 May 2010 at 7:31

GoogleCodeExporter commented 9 years ago 
This solution has 2 major drawbacks:
1) dependent jar-files, loaded with help of DC will be available only in Mapper 
and 
Reducer, but sometimes we need these jars in main method of Job
2) it will not work in LocalJobRunner mode

I think it will be better if we leave handling of <classpath> as it is now - we 
will 
pack all dependencies in jar file of Hadoop job

Original comment by vorl.s...@gmail.com on 31 May 2010 at 1:26

GoogleCodeExporter commented 9 years ago 
This bug is fixed in r312

Original comment by vorl.s...@gmail.com on 4 Jun 2010 at 1:34