Open sergeysolovev opened 7 years ago
Found the reason and possible fixes.
If the client sends the header Accept-Encoding
with non-empty value, e.g. gzip, deflate, br
(and it usually does), the production reverse proxy compresses response body, received from django, and sends back to the client. In this case the reverse proxy prepends an ETag strong validator, generated by django, with W/
prefix, to make it a weak validator and meet https://tools.ietf.org/html/rfc7232#section-2.1. That's how nginx works and that's right.
One possible solution is to avoid sending Accept-Encoding
from the client. But it's not always possible. For example, this header is a forbidden header name for fetch. The same applies for XMLHttpRequest. It means that this solution won't work for JavaScript clients.
Another solution is to make compression on the upstream side with django.middleware.gzip.GZipMiddleware
and remove the suffix ;gzip
from generated ETag value, like described here. In this case the reverse proxy keeps ETag untouched.
To see how it works, request this url twice from the browser:
https://swapi.now.sh/api/?format=json
The second request should return 304 NOT MODIFIED
, as expected. See my previous message for more details on how to test it.
PR is coming.
May I please take a look on your nginx config, I am struggling configuring if-none-match. I am new to this so it would be really helpful if I can compare the proxy_cache_key config part.
Thank you
swapi.co doesn't return 304 (Not Modified) on subsequent requests with
If-None-Match
request header set to the same value as ETag from the server responseSteps to reproduce (Chrome) Make sure that cache is not disabled in dev tools. Request the following url twice directly from the browser:
http://swapi.co/api/?format=json
, or executefetch('http://swapi.co/api/?format=json')
from console.The first response from the server would have
ETag
header like this:The second request to the server would have
If-None-Match
header with the same checksum:It is expected that the second response would have status code 304 (Not Modified) with empty body, since the checksum hasn't changed. But the actual status code is 200 with the same body as the first request's.
Dev environment The thing is, it works as expected in dev environment, given that it's being served over HTTP/1.1.
Possible reasons The first notable difference between prod and dev is that prod responds with weak ETag in the form
W/"<string>"
. It might have something to do with cloudflare, since they support ETag. I haven't investigated that further though.That I am sure about is that it has to do with weak ETags. Try to make the second request, mentioned above, through curl:
outputs
Then try to manually remove
W/
part from the checksum:outputs
It would be nice to have it working!