vinissimus / async-asgi-testclient

A framework-agnostic library for testing ASGI web applications
MIT License
160 stars 20 forks source link

streaming not working with newer versions of starlette #56

Open falkben opened 2 years ago

falkben commented 2 years ago

First, thank you for creating this! This package is the only testing utility I've found that can consume and test an asgi streaming response (fastapi/starlette).

Inside this package, with newer versions of starlette, I do see some failures with some of the streaming tests.

Starting in 0.13.3

Then in 0.13.4, the same test starts to hang, instead of outright fail.

In 0.13.5, this test starts to hang as well.

In the latest version of starlette, 0.21.0, test_request_stream hangs, while test_upload_stream_from_download_stream fails with the following error:

$ pytest async_asgi_testclient/tests/test_testing.py::test_upload_stream_from_download_stream

============================= test session starts ==============================
platform darwin -- Python 3.10.6, pytest-7.1.3, pluggy-1.0.0
rootdir: /Users/bfalk/repos/async-asgi-testclient
plugins: anyio-3.6.1, asyncio-0.19.0, cov-4.0.0
asyncio: mode=strict
collected 1 item

async_asgi_testclient/tests/test_testing.py F                            [100%]

=================================== FAILURES ===================================
___________________ test_upload_stream_from_download_stream ____________________

starlette_app = <starlette.applications.Starlette object at 0x104bf5750>

    @pytest.mark.asyncio
    async def test_upload_stream_from_download_stream(starlette_app):
        from starlette.responses import StreamingResponse

        async def down_stream(request):
            def gen():
                for _ in range(3):
                    yield b"X" * 1024

            return StreamingResponse(gen())

        async def up_stream(request):
            async def gen():
                async for chunk in request.stream():
                    yield chunk

            return StreamingResponse(gen())

        starlette_app.add_route("/download_stream", down_stream, methods=["GET"])
        starlette_app.add_route("/upload_stream", up_stream, methods=["POST"])

        async with TestClient(starlette_app) as client:
            resp = await client.get("/download_stream", stream=True)
            assert resp.status_code == 200
            resp2 = await client.post(
                "/upload_stream", data=resp.iter_content(1024), stream=True
            )
            chunks = [c async for c in resp2.iter_content(1024)]
>           assert len(b"".join(chunks)) == 3 * 1024
E           AssertionError: assert 1024 == (3 * 1024)
E            +  where 1024 = len(b'XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX...XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX')
E            +    where b'XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX...XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX' = <built-in method join of bytes object at 0x1030d4030>([b'XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX...XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX', b'', b''])
E            +      where <built-in method join of bytes object at 0x1030d4030> = b''.join

async_asgi_testclient/tests/test_testing.py:516: AssertionError
=========================== short test summary info ============================
FAILED async_asgi_testclient/tests/test_testing.py::test_upload_stream_from_download_stream
============================== 1 failed in 0.10s ===============================
masipcat commented 2 years ago

Hi @falkben

Thank you for your detailed report.

It seems that starlette doesn't like reading the request body after starting the response. I changed the test to consume the request body and then start the response and this seems to work fine (https://github.com/vinissimus/async-asgi-testclient/pull/57).

I also created this example app without asgi-testclient to be sure it's not a bug in the test client, and it seems that the problem is in starlette:

# example.py
from starlette.applications import Starlette
from starlette.responses import StreamingResponse

app = Starlette()

async def buffered(request):
    chunks = [chunk async for chunk in request.stream()]

    async def gen(chunks):
        for chunk in chunks:
            yield chunk

    return StreamingResponse(gen(chunks))

async def unbuffered(request):
    return StreamingResponse(request.stream())

app.add_route("/buffered", buffered, methods=["POST"])
app.add_route("/unbuffered", unbuffered, methods=["POST"])

"""
1. Create a test file
dd if=/dev/urandom of=./file count=1024 bs=1024

2. Run server
uvicorn example:app

3. Check buffered stream works
curl -v -X POST localhost:8000/buffered --data-binary @file --output -

4. Test unbuffered stream doesn't work (empty response)
curl -v -X POST localhost:8000/unbuffered --data-binary @file --output -
"""

I was going to open a discussion in Starlette but I found this one reporting a similar problem