valyala / fasthttp

Fast HTTP package for Go. Tuned for high performance. Zero memory allocations in hot paths. Up to 10x faster than net/http
MIT License
21.94k stars 1.76k forks source link

It is not safe to read all stream body to memory without a max size limit. #1765

Open Quons opened 7 months ago

Quons commented 7 months ago

https://github.com/valyala/fasthttp/blob/57b9352ad1cc93a0aaaa72b2130e03ace8a5b118/http.go#L427 I think it would be safe to stop reading the request body into memory and return an error when it exceeds the maximum request body size. Otherwise, it may lead to an out-of-memory (OOM) error when the request body is too large.

erikdubbelboer commented 7 months ago

Users using body streaming on the server side should use Request.BodyStream() to get an io.Reader and read as much as they need. It's not recommended to to call Request.Body() to then get the full body. I agree that we should probably maybe do something here to prevent this. I'm open to a pull request or suggestion on how to fix this.

byte0o commented 5 months ago

If you want to add a read data size limit to the Body function, can you abandon the Body function and only support BodyStream? BodyStream can satisfy the caller's control of the read data size.

gaby commented 4 months ago

@byte0o That would be a backward incompatible change, right?

byte0o commented 4 months ago

@gab Yes, you can first mark the Body function as deprecated

dojutsu-user commented 2 months ago

@erikdubbelboer I would like to start contributing to this. From the above discussion, it seems that we can mark the Body() function as deprecated but it is getting used at many places in the same file. We should probably replace those also, right?

erikdubbelboer commented 2 months ago

@dojutsu-user maybe you can have a look at changing it so that Request.Body() prints a warning using Server.Logger (so only when used in a server context).

dojutsu-user commented 2 months ago

@erikdubbelboer We'll have to inject the Server.Logger instance in the request, right? Currently, there's no logger that I can fine in the req.

erikdubbelboer commented 2 months ago

Yeah but keep it private, we don't want to expose that.