This PR updates ShuffleExchangeExec to carry more information about how much we can change the partitioning. For repartition(col), we should preserve the user-specified partitioning and don't apply the AQE local shuffle reader.
Similar to repartition(number, col), we should respect the user-specified partitioning.
This PR updates
ShuffleExchangeExecto carry more information about how much we can change the partitioning. Forrepartition(col), we should preserve the user-specified partitioning and don't apply the AQE local shuffle reader.Similar to
repartition(number, col), we should respect the user-specified partitioning.No
a new test
Closes #30432 from cloud-fan/aqe.
Authored-by: Wenchen Fan wenchen@databricks.com Signed-off-by: Wenchen Fan wenchen@databricks.com
Upstream SPARK-XXXXX ticket and PR link (if not applicable, explain)
What changes were proposed in this pull request?
Why are the changes needed?
Does this PR introduce any user-facing change?
How was this patch tested?