in this scenario x is not a partitioned column but y is a partitioned column:
select * from tbl where x = 'abc' and y = substr(x,1,2);
will it benefit from partition pushdown/dynamic filtering just as fast as below would?
select * from tbl where x = 'abc' and y = 'ab';
from david -->
In theory, the optimizer should be able to propagate the constant and push the implied expression on y to the connector, but it doesn't seem to do that today. However, the black box predicate is pushed to the connector, and the Hive connector will filter the partition names during planning, so the effect is nearly the same. (it still has to list all the partition names, whereas pushing the implied y expression would allow pushing that into the metastore)
in this scenario x is not a partitioned column but y is a partitioned column:
select * from tbl where x = 'abc' and y = substr(x,1,2);
will it benefit from partition pushdown/dynamic filtering just as fast as below would?
select * from tbl where x = 'abc' and y = 'ab';
from david --> In theory, the optimizer should be able to propagate the constant and push the implied expression on y to the connector, but it doesn't seem to do that today. However, the black box predicate is pushed to the connector, and the Hive connector will filter the partition names during planning, so the effect is nearly the same. (it still has to list all the partition names, whereas pushing the implied y expression would allow pushing that into the metastore)
cc: @electrum @rzeyde-varada @sopel39