search

spoddutur / spark-notes

https://spoddutur.github.io/spark-notes/
305 stars 138 forks source link

calc error #10

Open WF1a opened 5 years ago

WF1a commented 5 years ago

https://spoddutur.github.io/spark-notes/distribution_of_executors_cores_and_memory_for_spark_application

So, if we request 20GB per executor, AM will actually get 20GB + memoryOverhead = 20 + 7% of 20GB = ~23GB memory for us.

20GB + memoryOverhead = 20 + 7% of 20GB = ~22GB