Open oneonestar opened 1 month ago
simplifies the extension of the routing function, facilitates testing and verification
This is needed, thank you!
The overall philosophy looks good to me. I like that the current architecture is a specialization of the more general architecture here, so that we can provide built in plugins to current users and avoid breaking changes.
The label concept is appealing, and should allow more sophisticated routing decisions. Have you considered taking a bit more inspiration from kubernetes and using taints? For example you could taint a cluster like taints: ['oneMinuteMaxExecution']
, and a routing plugin could either decide that that select 1
can tolerate this, or the client could add a header like X-Trino-Gateway-Toleration: oneMinuteMaxExecution
. Otoh, maybe labels are sufficiently powerful.
How about a feature/requirement section that includes the following info.
Generally big +1 on this. I love the idea of modularizing the router policies and allowing them to stack with different precedence ordering.
Adding a couple of things for us to consider with this design:
ClusterStatsMonitor
which fetches cluster-level info. What if someone is writing a new plugin and wants to use some cluster info that is not currently available in ClusterStats
? Do the plugin authors have to get changes merged to ClusterStatsMonitor
first, then consume it? Or do we provide some mechanism for a routing plugin to request arbitrary information as part of ClusterStatsMonitor
? The latter may be very useful for extensibility. I can imagine a mechanism whereby a plugin can provide e.g. a Function<ProxyBackendConfiguration, Map<String, String>>
that can populate arbitrary key-value pairs of information.String
or Object
? Or, would we provide structured support for certain types of state?
Trino Gateway Routing Framework
Summary
Introduce Trino Gateway Routing Framework that simplifies the extension of the routing function, facilitates testing and verification. This allows routing logics to be implemented as plugins.
Motivation
The routing logic required can vary significantly depending on the use case. The types of workloads that need to be handled and the methods for operating Trino clusters differ for each user.
Although the gateway supports some level of customization of routing logic using MVEL, Java code may be preferred due to its compile-time checking and ease of testing.
Additionally, the current default routing logic is hard-coded, making it difficult to understand or modify according to specific needs. Modularizing the default routing logic would facilitate better testing and customization.
Goals
Non-goals
Remove or replace MVEL (currently supported by using Easy Rules). Defining routing logics using MVEL will continue to be supported.
Proposal
Architecture
The routing framework defines a set of APIs that allow routing logics to be implemented as plugins. These plugins can be chained in different order according to the configuration. Each plugin decides either to make a routing decision or to pass it to the next plugin.
Information can be passed from earlier plugins to later ones, allowing them to work together to form complex logic.
Lazy evaluation
Different plugin requires different information. Most of the information can be derived from the request. Lazy evaluation will be used to minimize the computation cost. For example,
Route by queryID
plugin needs a query ID to make a decision. The plugin try to get a query ID, which trigger a computation to try to extract query ID from the request. The computed result could be stored for other plugins to use later.(Below are WIP & random ideas)
Example 1: X-Trino-Routing-Group
Example 2: Route by User
Example 3: Load aware routing
Other possible plugins:
Other random thoughts: ClusterStatsMonitor could grep more info from Trino, which enables more routing plugin to be implemented. eg.
Label based routing: Higher flexibility than the current routing group logic. Trino1: label={routing_group: batch, version: 440, env: prod} Trino2: label={routing_group: batch, version: 450, env: staging} Trino3: label={routing_group: adhoc, version: 440, env: test} Trino4: label={routing_group: admin, version: 440, env: prod}
Weight based routing: Weight cluster and distribute queries according to the weights. Weight cluster and select the highest weight cluster.
Other things that might need to be aware of:
Ref: