search

erikdubbelboer / fasthttp

This fork has been deprecated!
MIT License
50 stars 12 forks source link

Set response reader for fasthttp #14

Closed dgrr closed 4 years ago

dgrr commented 6 years ago

Hello.

Fasthttp allows to set response reader of body like net/http package?

erikdubbelboer commented 6 years ago

You mean like https://godoc.org/github.com/erikdubbelboer/fasthttp#RequestCtx.SetBody ?

dgrr commented 6 years ago

No. I want to do in a client, to receive an amount of data greater than 10MB and don't store it in memory

erikdubbelboer commented 6 years ago

I'm afraid fasthttp doesn't support that. The only thing you can do is set MaxResponseBodySize but this will just abort the request once the size is reached.

dgrr commented 6 years ago

The main problem is that fasthttp load all content in memory buffer and then you can handle as you want. I think we can implement some io.Reader/Writer to read or write the body request/response. My final intention is to store and handle big files without storing in memory. Any proposal how to do that?

erikdubbelboer commented 6 years ago

Client uses Response.ReadLimitBody to read the response body. This method then allocates a bytebufferpool.ByteBuffer and calls readBody which then calls one of 3 method depending on the encoding of the body.

Right now readBody and the method it calls directly use ByteBuffer.B. If you could change these methods to accept an io.Writer/io.ReaderFrom combination of some sorts (that bytebufferpool.ByteBuffer already implements) you could add an extra argument to Client to provide your own (instead of the one readBody allocates).

dgrr commented 6 years ago

Um, and what about func GetBody() io.Reader as https://godoc.org/net/http#Request? We can implement something like this

erikdubbelboer commented 6 years ago

No that's not possible as Client already fetches the body in Client.Do. It needs to be either an io.Writer/io.ReaderFrom or callback function set somewhere. It could be in Client but this would mean all requests use the same function and wouldn't be really safe. Or it could be set in Request or Response. I'm not sure what is best and easiest.