(Question) Load Balancer/Reverse Proxy in ActiveJ

[ja3abuser]

Hi, are there any examples of load balancing/reverse proxy on ActiveJ?

[eduard-vasinskyi]

Hi, @wireguard-dev

Could you be more specific about what examples you are interested in?

Is it an example of ActiveJ with a third-party proxy? Or are you interested in how to set up an ActiveJ HTTP server as a reverse proxy?

[ja3abuser]

Hi, @wireguard-dev

Could you be more specific about what examples you are interested in?

Is it an example of ActiveJ with a third-party proxy? Or are you interested in how to set up an ActiveJ HTTP server as a reverse proxy?

Hi, @eduard-vasinskyi! I want to set up ActiveJ as a reverse proxy.

[eduard-vasinskyi]

@wireguard-dev ActiveJ was not designed with reverse proxy capabilities in mind. So, there are currently no examples for ActiveJ as a reverse proxy. We will consider adding those later.

But that does not mean that you cannot create your own reverse proxy out of ActiveJ HTTP components! We actually use a simple ActiveJ-based reverse proxy in some of our production projects.

At a basic level, to create a reverse proxy with ActiveJ you would need an AsyncHttpServer to receive requests and AsyncHttpClient to forward those requests to the actual server. You would need to create a new HttpRequest out of the HttpRequest received in AsyncHttpServer’s servlet. To do that you would need to copy the request URL, headers and re-stream HttpRequest’s body stream. After sending the copy of the original request to the HTTP client and receiving the response, you would need to copy this response and send it as the HTTP servlet’s response.

That is a basic idea. If you plan to implement the reverse proxy, there are some caveats, like the “Host” HTTP header, as that header should not be redirected as-is, etc.

[Vynbz]

Hi, I want to set up ActiveJ as a load balancer.

Do you have any ideas how to distribute the requests across multiple servers, as in ActiveJ RPC? Also, do you have any recommendations for increasing RPS that can be processed by the server without load-balancing? The issues of creating a Load Balancer/Reverse Proxy are very important, it would be great if you add examples for these two use cases to the documentation.

[Vynbz]

Hi, @eduard-vasinskyi, I will be glad for any response.

[eduard-vasinskyi]

Hi, @Vynbz

ActiveJ HTTP is a general-purpose HTTP framework. You can use it to create various HTTP applications, including reverse proxy, load balancer, etc. For load balancing you would need an HTTP server to handle requests and an HTTP client to forward requests to underlying servers.

The approach to configuring ActiveJ HTTP as a load balancer is similar to the reverse proxy approach that I discussed earlier. The only difference is that you will have a pool of servers rather than a single underlying server. Your code will have to select a server from a pool based on your load-balancing logic.

Another approach is to balance the load at the TCP level rather than the HTTP level. To do this, you will need to create your own AbstractReactiveServer or use SimpleServer, which is a simple implementation of a TCP server. The basic idea is to forward all incoming data from socket to socket (note, that this way you lose the ability to modify headers, etc.).

The code would probably look something like this (although, I haven’t tested it myself):

SimpleServer.builder(reactor, socketIn -> {
   InetSocketAddress address = chooseServer();
   TcpSocket.connect(reactor, address)
      .then(socketOut -> Promises.all(
         ChannelSuppliers.ofSocket(socketIn)
            .streamTo(ChannelConsumers.ofSocket(socketOut)),
         ChannelSuppliers.ofSocket(socketOut)
            .streamTo(ChannelConsumers.ofSocket(socketIn))
      ));
});

To increase RPS on a server without using load balancing, you can use ActiveJ Workers. You set up multiple worker servers and a single primary server that redirects connections to worker servers. Each worker server runs in a separate eventlooop (separate thread). The idea is similar to load balancing, but on a single machine. It is easy to set up multiple worker servers using the ActiveJ DI WorkerPoolModule. There is also an example of using an HTTP server with workers. You can look at the code of MultithreadedHttpServerLauncher to get an idea of how workers are configured.

[Vynbz]

Hi, @eduard-vasinskyi

Can I create a MiTM SimpleServer (that acts like CloudFlare) that adds an IP header before processing request to the socket? It should work with HTTPS requests. Can you tell me how I can decrypt HTTPS traffic, add the header and encrypt it again to send to my HTTPS server using ActiveJ?

[eduard-vasinskyi]

Hi, @Vynbz

Do you mean the IP header, i.e. the Internet Protocol header? The ActiveJ Network module works on the Transport layer of the OSI model, i.e. TCP/UDP. You cannot change IP headers, which are part of the network layer.

To decrypt HTTPS traffic on a server, you need to bind the server to be listening on addresses with SSL enabled. To do this, simply call AbstractReactiveServer.Builder#withSslListenAddresses instead of AbstractReactiveServer.Builder#withListenAddresses as you would normally do for unencrypted traffic. You need to additionally pass the SSLContext object and the Executor that is used to handle tasks required to manage handshakes.

[Vynbz]

Hi, @eduard-vasinskyi

I mean, I want to add my custom header with IP, because the servers only see the IP of my load balancer server. I started the server using SSL, but the requests were decrypted (as expected), so do I need to create a client to resend them over HTTPS?

[eduard-vasinskyi]

@Vynbz

If you plan to use SimpleServer, you can parse incoming requests manually and add/replace headers before sending them to the actual server. To make the out socket secure you need to wrap it using the SslTcpSocket#wrapClientSocket method.

Alternatively, you could use the HttpServer with HttpClient instead of SimpleServer. That way header parsing is done via HttpServer. The code for the server’s servlet would look something like this:

public AsyncServlet servlet(IHttpClient httpClient) {
   return request -> {
      InetSocketAddress address = chooseServer();

      // change URL, so that it corresponds to the chosen server
      String fullUrl = changeUrl(request.getFullUrl(), address);

      HttpRequest.Builder requestBuilder = HttpRequest.builder(request.getMethod(), fullUrl);
      for (Map.Entry<HttpHeader, HttpHeaderValue> entry : request.getHeaders()) {
         if (entry.getKey().equals(HttpHeaders.HOST)) {
            // change 'Host' header, so that it corresponds to the chosen server
            HttpHeaderValue host = changeHost(entry.getValue(), address);
            requestBuilder.withHeader(entry.getKey(), host);
            continue;
         }

         requestBuilder.withHeader(entry.getKey(), entry.getValue());
      }

      // Add custom headers
      requestBuilder.withHeader(HttpHeaders.of("my-header"), "some value");

      // Redirect request's body stream
      requestBuilder.withBodyStream(request.takeBodyStream());

      return httpClient.request(requestBuilder.build());
   };
}

To make HttpClient send encrypted requests, you need to create the client with SSL enabled: HttpClient.Builder#withSslEnabled and make sure that the URL has https schema.

[Vynbz]

Hi, @eduard-vasinskyi

I am experimenting with SimpleServer and noticed, that sending a ByteBuf with a response is ~3x slower than issuing a response from HttpServer.

Slower:

                    .to((serverReactor) -> SimpleServer.builder(serverReactor, socketIn -> {
                                        final String httpResponse = """
                                                HTTP/1.1 200 OK
                                                Connection: keep-alive
                                                Content-Type: text/plain; charset=utf-8
                                                Content-Length: 27

                                                Hello from the SimpleServer
                                                """;
                                        final ByteBuf byteBuf = ByteBuf.wrapForReading(httpResponse.getBytes(StandardCharsets.UTF_8));
                                        ChannelSuppliers.ofValue(byteBuf)
                                                .streamTo(ChannelConsumers.ofSocket(socketIn));
                                    })

Faster:

         ChannelSuppliers.ofSocket(socketIn)
            .streamTo(ChannelConsumers.ofSocket(socketOut)),
         ChannelSuppliers.ofSocket(socketOut)
            .streamTo(ChannelConsumers.ofSocket(socketIn))

Also tried socketIn.write(ByteBuf), but the result is just a little bit faster.

Maybe I should use another ChannelSupplier?

[eduard-vasinskyi]

@Vynbz

Many factors can affect server performance. It's hard to determine the actual bottleneck without the code. If you could provide the code that you used for testing as well as a tool you used to load the server, I could try to take a look.

The first thing that comes to mind is that HttpServer reuses keep-alive connections. Whereas, judging from your snippet, SimpleServer uses a new connection for each request. Managing connection creation and termination (especially with secure connection) requires several network round trips, so that may impact the performance.

Maybe you could use the HttpServer instead of the SimpleServer? As it is tuned for highload traffic and manages keep-alive connections, timeouts, etc…

[Vynbz]

You're right. I thought that with each request we open a new connection using TcpSocket.connect method, my idea was that by saving the response from the server, I can not request it again (not open the connection), but give it directly from the load balancer. Thanks for your answers, I have solved my problem.