Hang when writing to a Response via ContentSinkOutputStream

[adolski]

Jetty version(s) 12.0.14 and others in the 12.0.x series

Jetty Environment core

Java version/vendor (use: java -version)

openjdk version "23.0.1" 2024-10-15
OpenJDK Runtime Environment Temurin-23.0.1+11 (build 23.0.1+11)
OpenJDK 64-Bit Server VM Temurin-23.0.1+11 (build 23.0.1+11, mixed mode, sharing)

OS type/version

• macOS Sequoia 15.0.1 ARM64
• Rocky Linux 9.3 ARM64

Description While writing to a Response via the OutputStream returned from Content.Sink.asOutputStream(), the write will often hang while only partially written, and then time out. The more bytes written, the greater the chances of the hang. A few MB is enough to trigger it pretty frequently.

When instead using Response.write(), there is no problem.

How to reproduce?

I have prepared an SSCCE here: https://github.com/adolski/jetty-sscce/tree/main

See the main class for a minimal example.

[joakime]

The streams from Java (InputStream, OutputStream) are blocking APIs. When you use them from a Handler like you are doing, the stream blocking API can block operations of the thread doing the handling (if that happens, the handling is also blocked, causing your apparent hang).

You should be doing that operation in a separate thread, to not block the operations of the handling thread.

Here's a modification of your example to handle any arbitrary sized used of OutputStream in a different thread.

package handlers;

import java.io.IOException;
import java.io.OutputStream;

import org.eclipse.jetty.io.Content;
import org.eclipse.jetty.server.Handler;
import org.eclipse.jetty.server.HttpConfiguration;
import org.eclipse.jetty.server.HttpConnectionFactory;
import org.eclipse.jetty.server.Request;
import org.eclipse.jetty.server.Response;
import org.eclipse.jetty.server.Server;
import org.eclipse.jetty.server.ServerConnector;
import org.eclipse.jetty.server.handler.StateTrackingHandler;
import org.eclipse.jetty.util.Callback;

public class OutputStreamHandler
{
    private static final int HTTP_PORT     = 8182;
    private static final int RESPONSE_SIZE = 20000000;

    private static Server server = new Server();

    public static void main(String[] args) throws Exception {
        StateTrackingHandler trackingHandler = new StateTrackingHandler();
        trackingHandler.setHandlerCallbackTimeout(30000);
        trackingHandler.setHandler(new Handler.Abstract() {
            @Override
            public boolean handle(Request jettyRequest,
                                  Response jettyResponse,
                                  Callback callback) throws Exception {
                jettyRequest.getComponents().getExecutor().execute(() -> {
                    try (OutputStream os = Content.Sink.asOutputStream(jettyResponse)) {
                        // The more bytes, the better the chance of causing the problem
                        byte[] bytes = new byte[RESPONSE_SIZE];
                        os.write(bytes);
                    } catch (IOException e) {
                        callback.failed(e);
                    }
                    callback.succeeded();
                });
                return true;
            }
        });

        HttpConfiguration config    = new HttpConfiguration();
        HttpConnectionFactory http1 = new HttpConnectionFactory(config);
        ServerConnector connector   = new ServerConnector(server, http1);
        connector.setPort(HTTP_PORT);

        server.setHandler(trackingHandler);
        server.addConnector(connector);
        server.start();
    }
}

With the output on command line ...

$ ls -la foo.dat
ls: cannot access 'foo.dat': No such file or directory
$ curl --output foo.dat -vvvv http://localhost:8182/
*   Trying 127.0.0.1:8182...
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0* Connected to localhost (127.0.0.1) port 8182 (#0)
> GET / HTTP/1.1
> Host: localhost:8182
> User-Agent: curl/7.81.0
> Accept: */*
> 
* Mark bundle as not supporting multiuse
< HTTP/1.1 200 OK
< Server: Jetty(12.0.14)
< Date: Wed, 23 Oct 2024 13:16:03 GMT
< Transfer-Encoding: chunked
< 
{ [65428 bytes data]
100 19.0M    0 19.0M    0     0   137M      0 --:--:-- --:--:-- --:--:--  138M
* Connection #0 to host localhost left intact
$ ls -la foo.dat
-rw-rw-r-- 1 joakim joakim 20000000 Oct 23 08:16 foo.dat

[adolski]

Thank you @joakime! I completely overlooked that.