apache / incubator-gluten

Gluten is a middle layer responsible for offloading JVM-based SQL engines' execution to native engines.
https://gluten.apache.org/
Apache License 2.0
1.22k stars 437 forks source link

[GLUTEN-8010][VL] Don't generate native metrics if filter's remainingCondition is null #8011

Open zml1206 opened 1 day ago

zml1206 commented 1 day ago

What changes were proposed in this pull request?

(Fixes: #8010)

How was this patch tested?

Local test. Before PR:

image

After PR:

image
github-actions[bot] commented 1 day ago

https://github.com/apache/incubator-gluten/issues/8010

github-actions[bot] commented 1 day ago

Run Gluten Clickhouse CI on x86

github-actions[bot] commented 1 day ago

Run Gluten Clickhouse CI on x86

github-actions[bot] commented 1 day ago

Run Gluten Clickhouse CI on x86

zhztheplayer commented 1 day ago

Thanks!

Though it feels like a workaround since there is no real filter node in Velox query plan? Do we have other better options?

zml1206 commented 1 day ago

Though it feels like a workaround since there is no real filter node in Velox query plan? Do we have other better options?

How about set metricsUpdater to MetricsUpdater.None when getRemainingCondition is null? In this way, there is no metrics information.

image
zhztheplayer commented 22 hours ago

How about set metricsUpdater to MetricsUpdater.None when getRemainingCondition is null?

Sounds fair. Let's see if it works.

BTW I think one of the promising ways is to exclude the filter node from Spark query plan as well. Though I remember there were some issues with an relevant attempt.

zml1206 commented 22 hours ago

BTW I think one of the promising ways is to exclude the filter node from Spark query plan as well. Though I remember there were some issues with an relevant attempt.

We can't exclude the filter node, because it's output may differ from child's output.

github-actions[bot] commented 22 hours ago

Run Gluten Clickhouse CI on x86

github-actions[bot] commented 19 hours ago

Run Gluten Clickhouse CI on x86

zml1206 commented 17 hours ago

@zhztheplayer Can you help take a look again? Thanks.