valyala / fasthttp

Fast HTTP package for Go. Tuned for high performance. Zero memory allocations in hot paths. Up to 10x faster than net/http
MIT License
21.95k stars 1.76k forks source link

how should convert bytesBuffer into http message object。 #763

Open aoliang opened 4 years ago

aoliang commented 4 years ago

i learned code below。

func (req *Request) Read(r *bufio.Reader) error {
    return req.ReadLimitBody(r, 0)
}

param r is io buf reader which could be a connection or something else。 i have a scene: goroutine read connection which is a very low eventloop with a fixed size of buffer,i dont wanna chang it。maybe i just get part of http bytes,not a complete http bytes,like chunk。 i cannot use Read method up there。 i did some research,netty has byteToMessageDecoder、envoy use http-parser (c++)https://github.com/nodejs/http-parser can we have this design future?

erikdubbelboer commented 4 years ago

I'm not sure I understand? You only get one chunk of the response at a time but you want your handler to process this chunk already? I'm afraid that's not possible with fasthttp. Your handler will only be called when a full request is received.

aoliang commented 4 years ago

I'm not sure I understand? You only get one chunk of the response at a time but you want your handler to process this chunk already? I'm afraid that's not possible with fasthttp. Your handler will only be called when a full request is received.

yeah. netty has HttpRequestDecoder can do parse part bytes of http request. it has state design. sample code below

switch` (state()) {
        case SKIP_CONTROL_CHARS: {
            try {
                skipControlCharacters(buffer);
                checkpoint(State.READ_INITIAL);
            } finally {
                checkpoint();
            }
            // fall-through
        }
        case READ_INITIAL: try {
            String[] initialLine = splitInitialLine(lineParser.parse(buffer));
            if (initialLine.length < 3) {
                // Invalid initial line - ignore.
                checkpoint(State.SKIP_CONTROL_CHARS);
                return;
            }
            message = createMessage(initialLine);
            checkpoint(State.READ_HEADER);
            // fall-through
        } catch (Exception e) {
            out.add(invalidMessage(e));
            return;
        }
        case READ_HEADER: try {
            State nextState = readHeaders(buffer);
            checkpoint(nextState);
            switch (nextState) {
            case SKIP_CONTROL_CHARS:
                // fast-path
                // No content is expected.
                out.add(message);
                out.add(LastHttpContent.EMPTY_LAST_CONTENT);
                reset();
                return;
            case READ_CHUNK_SIZE:
                if (!chunkedSupported) {
.....
erikdubbelboer commented 4 years ago

Just parsing the request headers can be done with RequestHeader.Read. But parsing only one chunk of a multi chunk response is something that is not exported at the moment. In theory we could modify the body read methods and expose some of them: https://github.com/valyala/fasthttp/blob/38aa88ab52c4b6f61722e2e7809c17b6f7df2b2a/http.go#L1766 A pull request from someone who has more ideas for this would always be welcome.