Open AngeloCa opened 10 months ago
As explained in this post. The sorting order might not be preserved within the partition when a spill occurs.
In order to avoid extra transformation steps or increase RAM instances it would be nice to have spark updates to fix this issue.
As explained in this post. The sorting order might not be preserved within the partition when a spill occurs.
In order to avoid extra transformation steps or increase RAM instances it would be nice to have spark updates to fix this issue.