Disable buffering

[alanpoulain]

When streaming, the response sent to the client seems buffered by default. It would be great to have an option to disable it.

[mliezun]

Hi @alanpoulain what do you mean exactly? What's the current behavior that you're observing and what's the expected behavior?

It would be great to know how this is gonna be used so I can provide a better solution.

[alanpoulain]

I've made some updates and I cannot reproduce it easily anymore (because of https://github.com/mliezun/caddy-snake/issues/30), but each time some data are yielded it should be received instantly by the client. It's not the case: data are received by batch, that's why I think there is some buffering.

[alanpoulain]

For instance with this code:

import time
from fastapi import FastAPI
from fastapi.responses import StreamingResponse

app = FastAPI()

def generator():
    for i in range(5):
        yield f"data: foo\n\n"
        time.sleep(1)

@app.get("/stream")
async def stream_test() -> StreamingResponse:
    return StreamingResponse(generator(), media_type='text/event-stream')

And this script:

import requests
import time

t_start = time.time()
response = requests.get('https://localhost/stream', verify=False, stream=True)
for chunk in response.iter_lines():
    print(f"Chunk (time {int((time.time() - t_start) * 1000)}ms): {chunk}")

The result looks like this:

python test.py
Chunk (time 5262ms): b'data: foo'
Chunk (time 5262ms): b''
Chunk (time 5262ms): b'data: foo'
Chunk (time 5262ms): b''
Chunk (time 5262ms): b'data: foo'
Chunk (time 5262ms): b''
Chunk (time 5262ms): b'data: foo'
Chunk (time 5262ms): b''
Chunk (time 5262ms): b'data: foo'
Chunk (time 5262ms): b''

Whereas we should see something like this:

python test.py
Chunk (time 262ms): b'data: foo'
Chunk (time 262ms): b''
Chunk (time 1262ms): b'data: foo'
Chunk (time 1262ms): b''
Chunk (time 2262ms): b'data: foo'
Chunk (time 2262ms): b''
Chunk (time 3262ms): b'data: foo'
Chunk (time 3262ms): b''
Chunk (time 4262ms): b'data: foo'
Chunk (time 4262ms): b''

[mliezun]

Thanks for the script, that helped!

I made some updates to the PR, now the response body is flushed as soon as it's received and you should see the expected behavior.

[alanpoulain]

It works perfectly, thank you very much!! :tada:

[mliezun]

Im closing this as solved because I just merged the PR that fixes this.