Closed michael-pxr closed 11 months ago
In what scenario is it necessary to send such a large header to the WebP Server Go?🤔
The website, say example.com, has a large Cookie header. When visiting webp.example.com to request the webp server, the Cookie header is also sent by the browser. Our webp server is behind an nginx proxy and the status code 431 is prevented by not forwarding the Cookie header from nginx to the webp server.
Note, however, that this issue is not about the webp server returning 431. It is about the webp server not logging a request that results in status code 431.
Okay, I got the reason for this.
WebP Server Go is using https://github.com/gofiber/fiber as HTTP framework, it has a constant called ReadBufferSize
:
ReadBufferSize specifies the I/O buffer size in bytes for incoming messages.
It's reflecting the same variable in FastHTTP, with description below:
// Per-connection buffer size for requests' reading.
// This also limits the maximum header size.
// Increase this buffer if your clients send multi-KB RequestURIs
// and/or multi-KB headers (for example, BIG cookies).
//
// Default: 4096
ReadBufferSize int `json:"read_buffer_size"`
Since is by default 4096, a large cookie of more characters will fail on FastHTTP first, so Fiber and WebP Server Go won't be able to log this request.
A quick fix for this(if you can compile by yourself), is to change
var app = fiber.New(fiber.Config{
ServerHeader: "WebP Server Go",
AppName: "WebP Server Go",
DisableStartupMessage: true,
ProxyHeader: "X-Real-IP",
})
to something like(You can adjust ReadBufferSize to something bigger that fits your needs):
var app = fiber.New(fiber.Config{
ServerHeader: "WebP Server Go",
AppName: "WebP Server Go",
DisableStartupMessage: true,
ProxyHeader: "X-Real-IP",
ReadBufferSize: 8192,
})
in webp-server.go
file and then compile the server(https://docs.webp.sh/build/), we are planning to make it changeable by supplying ENV in future version, stay tuned.
This issue should be fixed in 0.9.12 now. You need to customize your own env READ_BUFFER_SIZE
Describe the bug A request to the webp server that contains a large
Cookie
header will return status code 431 - Request Header Fields Too Large. However, this request in not shown in the logs of the webp server.To Reproduce Start the webp server:
Send a request with a large
Cookie
header:where
printf 'a%.0s' {1..5000}
prints the character 'a' 5000 times.The webp server does not log this request.
Expected behavior I expect the webp server to log the request that returns 431 - Request Header Fields Too Large.
Screenshots and logs N/A
Environment (please complete the following information):
Additional context N/A