search

aws / sagemaker-spark-container

The SageMaker Spark Container is a Docker image used to run data processing workloads with the Spark framework on Amazon SageMaker.
Apache License 2.0
36 stars 74 forks source link

Improve memory management, raise driver memory limit which causes out… #82

Closed larroy closed 2 years ago

larroy commented 2 years ago

Improve memory management, raise driver memory limit which causes out of memory errors when results are large. Account for executor memory overhead when calculating max executor memory.