Design Notes: Multiplexing one Downstream with multiple Upstreams

[jamesmunns]

A common use case for reverse proxies is to accept multiple "groups" of incoming connections.

This could be:

• One Service that serves multiple apex domains, but proxies to different subsets of listeners:
  • We have a single server that serves both example.com and elpmaxe.com
  • Each of these "identities" should route to different group of upstream servers
• One Service that routes a single apex domain, but proxies different kinds of requests to different subsets of Listeners:
  • example.com/v1/* goes to one set of upstreams, while example.com/v2/* goes to another set of upstreams
  • or example.com/**/*.pdf goes to one set of upstreams, while example.com/**/*.html goes to another set of upstreams
• It could also be possible we want to "mix and match" reverse proxying and static file serving
  • example.com/static/* should be served directly by river
  • example.com/* should be proxied to the listners

This arises as we may want to have something that feels like multiple services, but all served through a single set of :443 or :80 ports.

The current BasicProxy (nor static file service) is not flexible enough to handle these cases - we have a single set of "downstreams", and a single set of "upstreams" - with no way to multiplex between the two.

For example, NGINX allows multiple servers to listen to a single listening port. From this article:

server {
    listen 443 ssl;
    server_name wiki.example.com;
    ssl on;

    location / {
        proxy_pass http://server02.example.com:8090/;
    }
}

server {
    listen 443 ssl;
    server_name sickbeard.example.com;
    ssl on;

    location / {
        proxy_pass http://server01.example.com:8081/;
    }
}

Note that BOTH servers are listening to port 443, but each have their own upstreams (server02 and server01, with different ports).

[jamesmunns]

Expanding on these thoughts and trying to clarify what we might want:

Right now, a service has these properties:

• It has a set of downstream listeners - these listeners control the addr, port, and TLS settings used
• It has a set of upstream connectors:
  • These connectors have shared properties, like load balancing, health checking, etc.
  • Each connector has specific properties, like TLS-SNI
• It has a set of path-control settings:
  • These include a set of filters, modifiers, and rate limiting applied to every request/response
• As a bonus: file-serving services have a slightly different lifecycle

From a first glance, it seems like we would like to have a "multiservice" instance with the following qualities:

• One set of downstream listeners, as the current BasicProxy entity acts today
• N sets of the following items, each gated by some kind of query (more on this later):
  • For proxy services:
    • A set of upstream connectors
    • A set of path-control settings
  • For file-serving services:
    • A set of configuration used for this

There are two main phases where we could make this decision:

• at the request_filter stage - this is the first hook in the system
• at the upstream_peer stage - this is currently where upstreams are selected

I'm inclined to focus on the request_filter stage:

1. This is already where the file-serving hooks operate: pandora-web-server only acts in this phase, and serves the entire connection
2. As this is the first phase, we could make the "selection" logic here, figuring out which "set" of resources to use for the remainder of the lifecycle

Implementation wise, I believe this would require definining a new kind of service that would contain a unified set of rules (maybe just domain + URI?) to select which "subservice" matches, and continue using that.

This probably requires some kind of dyn trait that describes subservices, and delegates to that. This will likely then require storing the subservice handle in the CTX field, and using that to dispatch any later requests appropriately.