Spark cluster stalls on EMR when using defaults

Yelp / mrjob

Run MapReduce jobs on Hadoop or Amazon Web Services

http://packages.python.org/mrjob/

Other

2.61k stars 587 forks source link

Closed coyotemarin closed 4 years ago

coyotemarin commented 4 years ago

Even in mrjob 0.7.0, it's still possible to launch a Spark cluster that's too small to run. Should find out what defaults work on the latest AMIs.

coyotemarin commented 4 years ago

Was unable to duplicate this. Maybe EMR quietly fixed their cluster configs? Closing for now.

88manpreet commented 4 years ago

I think you planned to close this. Closing.