search

oap-project / raydp

RayDP provides simple APIs for running Spark on Ray and integrating Spark with AI libraries.
Apache License 2.0
293 stars 66 forks source link

Print warning message when spark.dynamicAllocation.maxExecutors is not set #278

Closed kira-lin closed 1 year ago

kira-lin commented 1 year ago

Print a warning message if dynamic allocation is enabled but maxExecutors is not set. The reason for this is because spark might request too many executors and crash ray if it is not set.

carsonwang commented 1 year ago

Can you please also update the title and description of the PR?

carsonwang commented 1 year ago

LGTM