Added the ability to set a custom number of reducers by defining a function "num_reduce_tasks" in your script. Ex:
def num_reduce_tasks():
return 23
There may be a better way to do this; I copied the approach from "skip_local_output". But it Works Today, so there's that...
Extended the functionality that mreid added for counters to local jobs run with FileDriver.py. There's probably a better way to do this too, but this also works fine.
Fixed a typo in healthreportutils.py.
Added the ability to set a custom number of reducers by defining a function "num_reduce_tasks" in your script. Ex: def num_reduce_tasks(): return 23 There may be a better way to do this; I copied the approach from "skip_local_output". But it Works Today, so there's that...
Extended the functionality that mreid added for counters to local jobs run with FileDriver.py. There's probably a better way to do this too, but this also works fine.