Streaming output does not work with http1.1

kolaente commented 1 month ago

What happened?

I'm trying to build an app which streams content to the user. It seems to work when using http2 or 3, but not locally with http1.1. I have enabled the full duplex option in my caddyfile and read https://github.com/dunglas/frankenphp/issues/86.

Minimal example:

<?php
for ($i = 1; $i <= 3; $i++) {
    echo "Chunk $i\n";
    flush();
    usleep(1000 * 500);
}

The different chunks should be sent to the client one after the other, but they appear to be sent all at once.

I'm using Frankenphp with Laravel Octane.

Build Type

Docker (Debian Bookworm)

Worker Mode

Yes

Operating System

GNU/Linux

CPU Architecture

x86_64

PHP configuration

click to expand

Relevant log output

No response

dunglas commented 1 month ago

What client are you using? If it's curl, can you try again with the --no-buffer flag? Most other clients have similar flags. Also, be sure that you don't use any reverse proxy (such as nginx) because they can also have their own buffers.

withinboredom commented 1 month ago

Also, with http1.1, IIRC, remember to set encoding to chunked. There are probably a few other things to do too. Streaming was a somewhat dark art in the http1.x days.

kolaente commented 1 month ago

What client are you using? If it's curl, can you try again with the --no-buffer flag? Most other clients have similar flags. Also, be sure that you don't use any reverse proxy (such as nginx) because they can also have their own buffers.

Using curl with the flag, directly calling the endpoint running on frankenphp, without any other proxy in between.

Also, with http1.1, IIRC, remember to set encoding to chunked. There are probably a few other things to do too. Streaming was a somewhat dark art in the http1.x days.

How would I do that?

withinboredom commented 1 month ago

How would I do that?

header('Encoding: chunked');

kolaente commented 1 month ago

Oddly enough, with the header it finishes the request right away, without returning anything. Here's the full script I used:


<?php
header('Encoding: chunked');

for ($i = 1; $i <= 3; $i++) {
    echo "Chunk $i\n";
    flush();
    usleep(1000 * 500);
}

withinboredom commented 4 weeks ago

As mentioned, getting streaming working with http 1.1 is a dark art. Right now, you are attempting to get it working with curl (if I understand correctly), but then each browser also has quirks you need to do. I (personally) don't remember all these things, because http2 has been around for long enough for me to forget. I highly recommend the same unless you absolutely need 1.1 for some reason, and in that case, I recommend digging around on old blogs from the early '10s and '00s. Even then, people switched to websockets ASAP to avoid these shenanigans.

But, for your example, you end a chunk with \r\n not \n IIRC.

AlliBalliBaba commented 3 weeks ago

I've gotten chunking to work before, you have to do something like this (also explained here):

set_time_limit(0);                // making maximum execution time unlimited
ob_implicit_flush(1);            // Send content immediately to the browser on every statement which produces output
ob_end_flush();                  // deletes the topmost output buffer and outputs all of its contents
for ($i = 1; $i <= 3; $i++) {
    echo "Chunk $i\n";
    usleep(1000 * 500);
}

kolaente commented 3 weeks ago

Gotcha, thanks for all the explanations. I don't need streaming per se - as long as it works in production that's fine by me (and production runs with http/2 and /3 so that's not an issue). I stumbled upon this when I tried this in my dev environment, which only does 1.1.

dunglas / frankenphp