vibe-d / vibe.d

Official vibe.d development
MIT License
1.15k stars 284 forks source link

requestHTTP: response readRawBody's input stream never reports empty #1976

Open dhasenan opened 6 years ago

dhasenan commented 6 years ago

vibe.d 8.1.0 with dmd 2.076.0 on ubuntu 16.04.

If I call requestHTTP and try to read its body using the low-level interface, it hangs indefinitely:

shared static this()
{
    import vibe.d;
    requestHTTP("https://example.org",
    (scope HTTPClientRequest req)
    {
        req.method = HTTPMethod.GET;
    },
    (scope HTTPClientResponse resp)
    {
        logInfo("connected to remote host!");
        resp.readRawBody(delegate void (scope InputStream stream) scope {
            logInfo("have body");
            while (!stream.empty)
            {
                logInfo("stream says it is not empty");
                auto buf = new ubyte[stream.leastSize];
                logInfo("reading %s bytes", buf.length);
                stream.read(buf);
                logInfo("read %s bytes", buf.length);
                logInfo("finished append");
            }
            logInfo("finished body");
        });
        logInfo("finished request");
    });
    logInfo("outside request");
}

Actual output:

connected to remote host!
have body
stream says it is not empty
reading 606 bytes
read 606 bytes
finished append

At this point, it hangs forever. The call to stream.empty appears to hang forever.

readAllUTF8 works somehow, but this way of doing it doesn't.

s-ludwig commented 6 years ago

What you probably want is .bodyReader.readAll(), which will read the logical body contents as untyped bytes. readRawBody specifically circumvents any content or transfer encoding and basically requires the user to re-implement that part of the HTTP protocol in an appropriate way. .empty() will basically operate directly on the TCP connection, so that it will return only if further data arrives, or the connection gets closed.