Closed cermmik closed 6 years ago
Set spark.streaming.dynamicAllocation.enabled=true in the run_application.sh to allow Spark decide the correct number of resources.
spark.streaming.dynamicAllocation.enabled=true
To enable dynamic allocation it is necessary to modify the microbatch size to fit the window size for all applications.
Created branch dynamic-resourcess-allocation to update all applications.
merged in #79. Closing
Set
spark.streaming.dynamicAllocation.enabled=true
in the run_application.sh to allow Spark decide the correct number of resources.To enable dynamic allocation it is necessary to modify the microbatch size to fit the window size for all applications.