apache / ignite

Apache Ignite
https://ignite.apache.org/
Apache License 2.0
4.82k stars 1.9k forks source link

The rules related to CoreRules.UNION_MERGE in PlannerPhase#OPTIMIZATION should be completely disabled or carefully considered. #11656

Closed asdfgh19 closed 1 day ago

asdfgh19 commented 1 week ago

image2024-11-13_14-23-48 union_query.txt union_query_plan.txt

The above SQL has many UNION ALL and subqueries. If the UNION_MERGE rule is applied, an IgniteUnionAll operator with 9 child nodes will be generated. From the above figure, we can see that each child node has 7 possibilities, that is, 7 7 7 7 7 7 7 7 7 = 40353607. This will cause the plan space in CBO to explode, and directly cause the result HashSet in the TraitUtils#fillRecursive method to occupy a huge amount of memory, causing the Java program to be stuck due to GC.

The related rules should be disabled is:

  1. CoreRules.UNION_MERGE
  2. CoreRules.MINUS_MERGE
  3. CoreRules.INTERSECT_MERGE