Open svenakela opened 2 months ago
Wow! This is perhaps the most fully documented, step-by-step issue I've ever seen on GitHub. Thank you @svenakela!
I was easily able to reproduce your described issue, thanks to your instructions and repo with the pre-prepared docker-compose file (I doubt I would have had the time to look at this issue, otherwise.)
I made the following observations:
SecResponseBodyAccess
to Off
makes things work as expected: now that the proxy isn't interested in the response data it sends the chunks to the client as soon as they're available and "Loading..." is displayed.SecResponseBodyAccess
to Off
makes no difference. Either this setting isn't being respected or Coraza and/or Caddy will always wait for a full response before sending anything back to the client. I'm not sure which is the case, but I would guess maybe the latter.I hope this helps drive your issue forwards!
According to https://github.com/corazawaf/coraza-caddy/blob/c2e0fbdc9c648550df8b2466ee5bf86bebbf2494/interceptor.go#L89-L104 if the response body isn't accessible nor processable we skip buffering. Would you mind trying with https://github.com/jcchavezs/coraza-httpbin @RedXanadu ?
Wow! This is perhaps the most fully documented, step-by-step issue I've ever seen on GitHub. Thank you @svenakela! ... I made the following observations: ....
- With Coraza + Caddy, setting
SecResponseBodyAccess
toOff
makes no difference. Either this setting isn't being respected or Coraza and/or Caddy will always wait for a full response before sending anything back to the client. I'm not sure which is the case, but I would guess maybe the latter.
Thanks for the kind words.
SecResponseBodyAccess
made no difference at all. Disabling the SecRuleEngine makes the issue go away, and a clean Caddy installation is behaving as expected. I would think that Caddy is innocent. According to https://github.com/corazawaf/coraza-caddy/blob/c2e0fbdc9c648550df8b2466ee5bf86bebbf2494/interceptor.go#L89-L104 if the response body isn't accessible nor processable we skip buffering. Would you mind trying with https://github.com/jcchavezs/coraza-httpbin @RedXanadu ?
I haven't checked the code, but is there a guarantee that the tx
settings are set from the config?
How can I help to proceed with this lil' bug of ours?
I can confirm that this does not seem to be related to Caddy.
Here's a minimal example to reproduce the issue:
package main
import (
"github.com/corazawaf/coraza/v3"
txhttp "github.com/corazawaf/coraza/v3/http"
"net/http"
"time"
)
func main() {
cfg := coraza.NewWAFConfig().WithDirectivesFromFile("coraza.conf") // SecRuleEngine Off = unbuffered, SecRuleEngine On = buffered
waf, _ := coraza.NewWAF(cfg)
http.ListenAndServe(":8000", txhttp.WrapHandler(waf, http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
flusher, _ := w.(http.Flusher)
w.WriteHeader(http.StatusOK)
w.Write([]byte("Hello "))
flusher.Flush()
time.Sleep(5 * time.Second)
w.Write([]byte("world!"))
})))
}
It seems that the ResponseWriter is always wrapped, and the "response processor" outputs the buffered response? https://github.com/corazawaf/coraza/blob/main/http/interceptor.go#L140-L147 https://github.com/corazawaf/coraza/blob/main/http/middleware.go#L163-L171 https://github.com/corazawaf/coraza-caddy/blob/main/coraza.go#L133-L140
Is it really necessary to wrap the ResponseWriter, If tx.IsResponseBodyAccessible() && tx.IsResponseBodyProcessable()
is false?
We're interested in disabling response buffering when SecResponseBodyAccess Off
is set, too, and there is a related issue in the coraza-caddy project.
Description
NextJs uses streams to send data asynchronously to web clients.
When a NextJs application is deployed behind a Coraza WAF extended Caddy server, streamed responses are hindered by the SecRuleEngine. If there for example is a spinner that NextJs wants the browser to render until data is fetched in the background, it will not show up and the browser will be dead silent until the data is coming through.
I've tried to disable everything body access, I've tried to write a rule that allows any response. Nothing helps except disabling
SecRuleEngine
completely.Steps to reproduce
We've setup an example repo that reproduces the problem. The project consists of a small NextJs application behind a Caddy server. The issue is appearing by default in this repo.
https://github.com/svenakela/coraza-streaming-issue
Disable
SecRuleEngine
and the app works as expected.Expected result
In this example project the bottom text should change to Loading... until data is sent by the app server.
Actual result
No textual change at all. The bottom text only change when the response is ready in the app server.