Luigi is a Python module that helps you build complex pipelines of batch jobs. It handles dependency resolution, workflow management, visualization etc. It also comes with Hadoop support built in.
Apache License 2.0
17.71k
stars
2.39k
forks
source link
scheduler prune_on_get_work configuration parameter is not documented #3295
There is a
prune_on_get_workconfiguration parameter in the scheduler that does what it says, pruning tasks, workers, and emails upon theget_workrpc call. However, this parameter is not listed in the documentation.