search

apache / pinot

Apache Pinot - A realtime distributed OLAP datastore
https://pinot.apache.org/
Apache License 2.0
5.42k stars 1.27k forks source link

Multistage join queries and adaptive routing compatibility #14107

Open dang-stripe opened 3 hours ago

dang-stripe commented 3 hours ago

We are testing some multistage queries with joins and adaptive routing turned on on a table using 3 replicas, 30 servers each. We joined against another much smaller table 90 replicas, 1 instance each on the same tenant. This led to all queries for the smaller table to only be handled by 2 of the 90 servers. This happened with adaptive routing on.

We tried turning it off, and we saw queries start routing to the other 90 servers. It seems like there might be some incompatibility or undefined behavior between multistage and adaptive routing.

If multistage + adaptive routing are not compatible, is it possible to disable adaptive routing for multistage queries by default? Or is this a bug in the code?

cc @Jackie-Jiang @jadami10 @priyen-stripe

dang-stripe commented 3 hours ago

@priyen-stripe could you include a sample query that was seeing this behavior?