search

mozilla / mozilla-reports

Repository for public analyses.
https://mozilla.report
5 stars 12 forks source link

Document issues with using coalesce in Spark #34

Closed harterrt closed 7 years ago

fbertsch commented 7 years ago

This behavior is by design, but it's definitely worth having this documented. [0] is insightful in understanding why Spark behaves this way.

[0] http://apache-spark-user-list.1001560.n3.nabble.com/partitions-coalesce-and-parallelism-tp8189p8263.html