xkrogen
commented
3 weeks ago We talked a bit about making the GW more of a "full proxy" in one of the recent GW dev syncs. It potentially unlocks a lot of new capabilities.
I like the idea you've proposed here of embedding this state into the client instead of storing it on the GW side. Tracking when a query has finished, and thus its state can be cleaned up, is an annoying process. Right now we just have a periodic task, every 2 hours, to clear our query records older than a configurable time window (but that query may actually still be running!): https://github.com/trinodb/trino-gateway/blob/f50b09d5f81ccf5c72efce345a4235727c879c06/gateway-ha/src/main/java/io/trino/gateway/ha/persistence/JdbcConnectionManager.java#L72-L82
Moving it to the client is in line with Trino philosophy in general, IMO, like how we implement session properties and prepared statements on the client-side.
For the UI, I think as you said, we could do a fan-out that pulls query results from each backend ... That also has the benefit of not having two copies of the same data (query IDs / query history stored on both GW and Coordinator).
Curious to hear what others think, but personally at first pass I like the idea. One thing we should consider is whether this would make it harder to implement other new functionality in the future.
shk3
oneonestar
Hi folks,
Have we considered rewriting next-uri and info-uri directly from the responses in order to achieve query-level sticky routing?
The idea is kinda similar to Trino Proxy, where Trino Gateway proxies all requests. Then we bind the URLs in the following ways:
/v1/...: handled by the current logic and proxied to one of the Trino coordinators based on whatever load balancing algorithms of choice. In the response, we rewrite the next-uri / info-uri to something like/backend1/v1/...that is directly pointing to the particular backend (or rewrite the X-Forwarded-* headers to achieve the same goal)./backend1/v1/...: directly proxied to backend 1 (but block POST to/v1/statementso that queries can only be submitted via the load balancer);/backend2/v1/...: directly proxied to backend 2 (but block POST to/v1/statementso that queries can only be submitted via the load balancer);With this approach, for query-level sticky routing, we don't need to track which backend each query id gets assigned to. Instead, such assignment is retained on the client side.
The caveat is that for the Trino UI, we would need to develop a way for users to do a combined search queries across all backends as well as a summary of all backend's stats.
Has this approach been considered in the past? We could eliminate the dependency on the databases / caches. If cross-regional networking could be a concern, we could even change the URLs with different domains to avoid inter-regional proxying.
I know Trino Gateway's architecture is pretty much set, so it's not necessarily something we have to do now, but mostly a discussion just in case later on it's needed.
George