search

dharmeshkakadia / presto-hdinsight

Presto on Azure HDInsight
8 stars 24 forks source link

Why presto query max memory per node is $(($(($memory/1706))-1))GB #15

Closed betarabbit closed 7 years ago

betarabbit commented 7 years ago

Sorry, I don't know if this is the right place to ask a question, please feel free to close the issue as needed.

Would you mind explain more about the following config, please? Why max memory per node equals to (yarn-node-memory / 1706) - 1 GB? Why not 1024? Why it has to minus 1 GB then?

"site.global.presto_query_max_memory_per_node": "$(($(($memory/1706))-1))GB"

Thanks in advance.

dharmeshkakadia commented 7 years ago

1706=1024/0.6 memory/1706 = memory*0.6/1024

0.6 because the recommended memory config is 60% of the memory 1024 to convert it to GB

Also, -1 GB for leaving some memory on each node for safety.

betarabbit commented 7 years ago

Thank you so much for your explanation.