Open xitep opened 7 years ago
Do we want to incorporate a fixed dependency on YARN? I think that whether the flow runs on YARN or any other system is dependent on the settings of the executor - i.e. you can run euphoria flow on flink or spark without YARN, if you configure the executor to run in standalone mode. It would of course be desirable to be able to run multiple flows with multiple executors from single driver code.
It would be desirable to run a program consisting of multiple flows on different executors. Example:
The idea is that euphoria executors be independent of the execution engine launchers and be able to allocate their own YARN containers from within which each would operate their own execution engine.