Execute DAGs via Dask

[OliverEvans96]

Pros:

• pip install
• python 3
• resource usage visualizations (#26)
• Fast, minimal overhead
• Probably quite flexible

Cons:

• Dask is really made for parallelizing computations, not full workflows.
  • Although @mrocklin wrote a nice comparison between Dask and Celery that might be useful
• No native support for command line execution

[OliverEvans96]

It looks like executing workflows via DAGs also gains us the ability to run tasks over SSH. I think this is a really nice option.

Also, see the worker tutorial.

In dask, a worker can have multiple cores. I'm not sure whether this concept carries over to fireworks.