trinodb / trino

Official repository of Trino, the distributed SQL query engine for big data, formerly known as PrestoSQL (https://trino.io)
https://trino.io
Apache License 2.0
10.55k stars 3.03k forks source link

Pushdown implied expression to metastore #2912

Open tooptoop4 opened 4 years ago

tooptoop4 commented 4 years ago

in this scenario x is not a partitioned column but y is a partitioned column: select * from tbl where x = 'abc' and y = substr(x,1,2);

will it benefit from partition pushdown/dynamic filtering just as fast as below would? select * from tbl where x = 'abc' and y = 'ab';

from david --> In theory, the optimizer should be able to propagate the constant and push the implied expression on y to the connector, but it doesn't seem to do that today. However, the black box predicate is pushed to the connector, and the Hive connector will filter the partition names during planning, so the effect is nearly the same. (it still has to list all the partition names, whereas pushing the implied y expression would allow pushing that into the metastore)

cc: @electrum @rzeyde-varada @sopel39

findepi commented 4 years ago

This should be handled as part of effective predicate extraction / constant folding.

tooptoop4 commented 4 years ago

@raunaqmorarka do u know which classes to change for this?