feat(network): support buffering transfer-encoding: chunked

[H4ad]

Closes #165

[codecov[bot]]

Codecov Report

All modified and coverable lines are covered by tests :white_check_mark:

Comparison is base (7727a94) 0.00% compared to head (f19ffd1) 99.93%.

Additional details and impacted files

```diff @@ Coverage Diff @@ ## main #167 +/- ## ========================================= + Coverage 0 99.93% +99.93% ========================================= Files 0 64 +64 Lines 0 7771 +7771 Branches 0 658 +658 ========================================= + Hits 0 7766 +7766 - Misses 0 5 +5 ```

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

[huntharo]

I cannot positively tell if this is going to work.

I'm looking for the place where it reads the first chunk length, skips the trailing \r\n after that chunk length, then copies the bytes of the specified length after that, then reads the next chunk length, skips the trialing \r\n again, reads the byte length specified, etc. I can't find that place but I might just be overlooking it.

I don't think we need to set the content-length header in the response because the Lambda will do that when we give it the body string or base64 encoded body value.

I'm also confused as to why there is still an option to throw an exception on chunked encoding for API Gateway V1 since we should always be stripping out the chunked response encoding?

But... it looks really good and either perfect or very close!

[H4ad]

I'm looking for the place where it reads the first chunk length, skips the trailing \r\n after that chunk length, then copies the bytes of the specified length after that, then reads the next chunk length, skips the trialing \r\n again, reads the byte length specified, etc. I can't find that place but I might just be overlooking it.

The chunk came in the following order:

HTTP/2 200 
server: awselb/2.0
date: Mon, 01 Jan 2024 05:58:45 GMT
content-type: text/plain
content-length: 65
x-powered-by: Express
\r\n19

\r\n

INITIAL PAYLOAD RESPONSE\n

17

\r\n

FINAL PAYLOAD RESPONSE\n

0\r\n\r\n

I basically need to ignore the first chunk, skip the second, take the third chunk, and then skip the next 2 chunks, and keep going until I find the endChunk.

The chunks will always be sent size, delimiter, and then data.

The code that handles this algorithm is https://github.com/H4ad/serverless-adapter/pull/167/files#diff-4f134ef86a335c260c3933721a28831071072ecafbafb3252e42cc2972d5bbf2R80-R84 Is the same logic I use for my Streaming Response support.

And from the tests, I worked without issues.

I'm also confused as to why there is still an option to throw an exception on chunked encoding for API Gateway V1 since we should always be stripping out the chunked response encoding?

Because it's a breaking change, and I don't want to release a major version, so I will just keep the same behavior and let the people explicitly use that option to enable the support, in the next major version, I can invert the value and keep that throw as false.

[huntharo]

Thanks for the details! Sounds good to me.

[H4ad]

Thanks for the review.