Open ggevay opened 1 year ago
Note that this issue will probably be easier to tackle after https://github.com/MaterializeInc/materialize/issues/20746.
Btw. one can also ask what should happen when a window function is inside a table function or vice versa.
This seems related to https://github.com/MaterializeInc/materialize/issues/20533
Related to https://github.com/MaterializeInc/materialize/pull/22964/files, i.e., that window functions are not supported in table function arguments.
Simplified example of the error a customer ran into:
select distinct unnest(array_agg(x)) from (values (1), (1), (2)) x;
Error: aggregate functions are not allowed in table function arguments (function pg_catalog.array_agg)
This appears to be a somewhat common technique to deduplicate an array in Postgres.
So, this query starts from individual values as an input, makes an array, blows up the array, and then does a distinct. It seems to me that the unnest(array_agg(x))
is a no-op here.
Does the user start from arrays? If yes, then maybe the nesting is in the other direction, i.e., array_agg(unnest(...))
? (But unfortunately, we don't support this nesting direction either at the moment, I think.)
It's a no-op here yes. I'm gonna guess that that example has some actual logic that got simplified away for a minimum repro.
There are some nuances lost from the customer’s use case for sure. Their example uses the context from the outer query akin to a lateral join. I think when we see this pattern it almost always means there is a more efficient rewrite, probably involving computing aggregates in a separate view/CTE and then joining back in.
We ended up finding a better rewrite for this particular application that involved using a MAX to determine what label should be applied to a group of records (see slack)
I’d say this customer is unblocked on this particular usage, and I’m not sure if there is any optimization lesson to be learned here. It’s probably good for now that this returns an error because it seems likely we should always be doing something other than unnesting an array_agg.
The first problem from the issue description ("a query has both a table function and an aggregate") occurred at a user today: https://materializeinc.slack.com/archives/C04SN4BEQ0N/p1724053612520059?thread_ts=1723657127.783599&cid=C04SN4BEQ0N
We could at least make the error msg nicer.
What version of Materialize are you using?
26b5206857bdd1186a066759fbce863ca6d30985
What is the issue?
When a query has table functions, window functions, and aggregates, then Postgres evaluates them in the following order:
We have two problems:
We bail out when a query has both a table function and an aggregate
Materialize:
Postgres:
When a query has just a table function and a window function, then we evaluate these in the opposite order from Postgres
Materialize:
Postgres:
Slack discussion: https://materializeinc.slack.com/archives/C02FWJ94HME/p1691091394842719