Open natekupp opened 5 years ago
cc @sryza - at some point we should revisit the GCP Dataproc stuff to use the new step launcher machinery
Renamed this issue to make it more specific to the remaining work
Probably makes sense to have an issue per platform, e.g. a separate issue if we want Azure HDInsight integration
cool, sounds good on all of the above!
Wonder if there is any update one this request?
Right now IIRC EMR, Dataproc are factory solids—we probably want these to be provisioned as resources so that you can seamlessly move workloads from local to remote Spark clusters