search

prebid / prebid-server

Open-source solution for running real-time advertising auctions in the cloud.
https://prebid.org/product-suite/prebid-server/
Apache License 2.0
423 stars 718 forks source link

Handle GZIP compression at the OpenRTB endpoints #2438

Closed sag-celejd closed 1 year ago

sag-celejd commented 1 year ago

The current endpoints do not seem to support any type of compression for incoming requests. Looking at the code naively e.g. in auction.go, a quick implementation looks somewhat simple:

Check the Content Encoding header in the parseRequest method and decompress if needed. Maybe even implement a gzip.Reader pool to keep GC pressure low.

Is there anything speaking against this approach - maybe I'm missing something. I would be willing to prepare a pull request if this seems fine.

SyntaxNode commented 1 year ago

We support gzip compression for incoming requests. This feature is disabled by default. To enable it, set the enable_gzip config value to true.

bretg commented 1 year ago

So sounds like this is a docs issue... adding that label.

sag-celejd commented 1 year ago

Thanks for the response.

Can you guys point me to where you actually handle incoming compressed requests in the code? I'm only seeing that https://github.com/NYTimes/gziphandler is being used to gzip the response.

I am aware of enable_gzip but that only seems to be relevant for the bidder adapters.

Maybe I expressed myself poorly. I might be doing something wrong but what I'm saying is that e.g. a request like so does not seem to be supported: echo '{"foo": "bar"}' | gzip | curl -v -i --data-binary @- -H "Content-Encoding: gzip" --request POST https://foo.bar/openrtb2/auction

SyntaxNode commented 1 year ago

I'm only seeing that https://github.com/NYTimes/gziphandler is being used to gzip the response.

Huh. It seems I misremembered that the NYTimes handler just handles the response, not the request.

I would be willing to prepare a pull request if this seems fine.

Sounds good to me. Please still utilize the LimitedReader so we have protection from large requests and from zip bombs.

josepowera commented 1 year ago

To support incoming GZIP compressed PBS requests would be also useful in: -server 2 server traffic - in SSAI and DOOH applications.
-in client side traffic to lower bandwidth (mobile) data transfer.

However gzip decompress could be also most probable done in loadbalancer before PBS.

SyntaxNode commented 1 year ago

Implemented in PBS-Go 0.259.0.