search

apache / pinot

Apache Pinot - A realtime distributed OLAP datastore
https://pinot.apache.org/
Apache License 2.0
5.42k stars 1.27k forks source link

[Adaptive Server Selector] support for strictReplicaGroup routing strategy #12507

Open MeihanLi opened 7 months ago

MeihanLi commented 7 months ago

It seems currently Adaptive Server Selector is partially supported for strictReplicaGroup. But it breaks the assumption that by default, for strictReplicaGroup routing, queries are routed to servers from 1 replica group only.

I tested and figured out that for an upsert table with 2 replica groups, queries can be routed to servers from 2 replica groups depending on the server performance.

For example, a tenant has 4 servers, the table has 2 partitions, 2 replicas, 2 replica groups. For select count(*) query, it can pick serverA and serverD from 2 different RGs or serverA and serverB from 1 RG to serve the query.

RG assignment:
RG 0: serverA, serverB
RG 1: serverC, serverD
segments are assigned as below:
partition 0     serverA              serverC
partition 1     serverB              serverD

Are there any other concerns on turning on adaptive server selector for strictReplicaGroup routing tables except that? Also I was wondering if anyone is looking into supporting numReplicaGroupsToQuery with Adaptive Server Selection. What's the plan for it?

MeihanLi commented 7 months ago

cc: @vvivekiyer @ankitsultana

vvivekiyer commented 7 months ago

AdaptiveServerSelection is not support for strictReplicaGroup. It's only supported in balanced and ReplicaGroup.

The current solution ranks servers and picks the best set of servers. For strictReplicaGroup, we might need a solution that ranks replicaGroups before picking the best one.

ankitsultana commented 7 months ago

@vvivekiyer if one enables adaptive server selector on a broker that serves Upsert tables, what would happen? Would it lead to incorrect results?