Hydrospheredata / mist

Serverless proxy for Spark cluster
http://hydrosphere.io/mist/
Apache License 2.0
326 stars 68 forks source link

exclusive worker mode as default #513

Closed dos65 closed 6 years ago

dos65 commented 6 years ago

Before cutting 1.0.0, I think it would be better to slightly change context default settings. Shared mode fits more for already well-tested functions to achieve better performance. At the start, it's better to use exclusive mode for development and testing because of #437. Increasing parallelization level should be also explicitly done from the user side because it depends on spark cluster setup.