All responses truncated to exactly 8192 bytes

[NoiSek]

Possibly related to #14

I'm running into an issue where using the provided BrotliFilter with default settings will cause all responses to be truncated to a Content-Length of exactly 8192 bytes.

If you adjust the parameters, even setting the compression quality to 0 still causes all routes that match that filter to be truncated.

I've looked through the jbrotli source and I can't see what could be causing this, but there aren't any headers being applied on my own end and the output is normal when the filter is not in use or isn't applied to a route.

Is there any known reason why this could be happening?

[NoiSek]

Similar, but likely unrelated to: https://bz.apache.org/bugzilla/show_bug.cgi?id=44494

Running Environment: Java - 7.0 Tomcat - 8.0.43

[NoiSek]

Forcing the pre-filtered response to attach an accurate Content-Length results in a response with no Content-Length once filtered, rather than a Content-Length of 8192. I'm unsure if this is the intended result-- I would have expected it to be either persisted or altered.

Is it possible that there is a buffer within BrotliServletFilter that isn't being flushed properly? Out of order or at the wrong time? 8192 is a very suspicious number.

[NoiSek]

After a few days of tinkering, the only thing that I've been able to confirm for certain is that:

1. The response is fully formed at the time jBrotli receives it
2. The buffersize of the underlying Brotli compressor is 65536 bytes
3. The resulting output is still exactly 8192 bytes

Wrapping the various writers in BufferedWriter / BufferedStreamWriter instances with an appropriately corresponding buffersize hasn't had a noticeable effect, which makes me think the issue is perhaps deeper than the servlet itself (some fundamental timing error perhaps). Changing the order in which buffers are flushed has also not had a noticeable effect.

It would be helpful if someone was able to reproduce the issue in a sterile environment, I don't have enough experience with Java to say where exactly this problem lies or to debug it properly.

Is it actually possible to examine the contents of the various OutputStreams at runtime?

[thelittlefireman]

I've got the same issue but i don't have time to investiguate. But after some research i found that :

There is a similated issue in GZipCompression, the "content-lenght" header have'nt the right value : Maybe https://github.com/MeteoGroup/jbrotli/blob/4d003fdabb9eff4aa1eeddb159ef28fcbf9f279b/jbrotli-servlet/src/main/java/org/meteogroup/jbrotli/servlet/BrotliServletOutputStream.java#L82

could be changed like this : http://www.java2s.com/Code/Java/File-Input-Output/GZIPFilterresponsestreamandResponseWrapper.htm

public void close() throws IOException {
    if (closed) {
      throw new IOException("This output stream has already been closed");
    }
    gzipstream.finish();

    byte[] bytes = baos.toByteArray();

    response.addHeader("Content-Length", 
                       Integer.toString(bytes.length)); 
    response.addHeader("Content-Encoding", "gzip");
    output.write(bytes);
    output.flush();
    output.close();
    closed = true;
  }

OR maybe it could be the nginx reverse-proxy : https://github.com/request/request/issues/1418 proxy_buffering off

please keep me informed 😀

[NoiSek]

I'm looking into this again and strangely, setting a Content-Length WILL alter the size of the response, but always return exactly 8192 uncompressed characters every time.

e.g.

Enforced Content Length	Actual Response Size
12,000	11.9kb
15,000	14.9kb
None	390.1kb

I would say that this might have something to do with BrotliServletResponseWrapper.setContentLengthLong being commented out but the GZip servlet which I assume this is based on functions completely as expected with the same function disabled.

What could cause the filter to return less data but still have exactly 8192 characters in the end? I have to think that this bug comes down to an unflushed buffer, but I don't think that the issue is in the ServletFilter anymore, I think the issue is a layer deeper than that within the actual implementation of JBrotli. I can't verify that manually or I would.