search

oap-project / gazelle_plugin

Native SQL Engine plugin for Spark SQL with vectorized SIMD optimizations.
Apache License 2.0
256 stars 77 forks source link

Native-sql-engine of OAP-1.1.1 cannot pass q64.sql and q95.sql when disable spark.oap.sql.columnar.preferColumnar and set spark.oap.sql.columnar.joinOptimizationLevel equal to 6. #349

Open haojinIntel opened 3 years ago

haojinIntel commented 3 years ago

When we disable spark.oap.sql.columnar.preferColumnar and set spark.oap.sql.columnar.joinOptimizationLevel equal to 6. The query plan of continuous joining of q64.sql cannot successfully go back to vanilla spark. Using the same configuration on OAP-1.1 has no issue.

haojinIntel commented 3 years ago

@zhouyuan @weiting-chen @zhixingheyi-tian Please help to track the issue. Thanks!

weiting-chen commented 3 years ago

The error for q65.sql shows

  1. Error when running OnHeap memory to ArrowWritable memory
  2. No columnar to row phase in DAG