search

willnorris / imageproxy

A caching, resizing image proxy written in Go
Apache License 2.0
3.55k stars 483 forks source link

Why is it not using cache? #416

Open muxa opened 5 months ago

muxa commented 5 months ago

I've configureD imageproxy WITH with caching to disk:

/app/imageproxy -addr 0.0.0.0:8088 -cache /tmp/imageproxy -verbose

When I request the image, it's downloaded, stored on disk, resized and the resized image is also stored on disk (I can see the files in the data folder).

However in the logs I see that cache was not used when I request the same image again. Note that the image url I'm requesting contains query string parameters.

imageproxy listening on 0.0.0.0:8088
2024/06/04 09:25:52 fetching remote URL: https://REDACTED/index.php?gf-download=2024%2F06%2Fimage123.jpg&form-id=47&field-id=7&hash=0afa966d17976e112816ada2cc6b9275
2024/06/04 09:25:56 request: {Method:GET URL:https://REDACTED/index.php?gf-download=2024%2F06%2Fimage123.jpg&form-id=47&field-id=7&hash=0afa966d17976e112816ada2cc6b9275#1000x0 Proto:HTTP/1.1 ProtoMajor:1 ProtoMinor:1 Header:map[Accept:[image/*] User-Agent:[willnorris/imageproxy]] Body:<nil> GetBody:<nil> ContentLength:0 TransferEncoding:[] Close:false Host:REDACTED Form:map[] PostForm:map[] MultipartForm:<nil> Trailer:map[] RemoteAddr: RequestURI: TLS:<nil> Cancel:<nil> Response:<nil> ctx:0xc000038020} (served from cache: false)
2024/06/04 09:25:58 fetching remote URL: https://REDACTED/index.php?gf-download=2024%2F06%2Fimage123.jpg&form-id=47&field-id=7&hash=0afa966d17976e112816ada2cc6b9275
2024/06/04 09:26:01 request: {Method:GET URL:https://REDACTED/index.php?gf-download=2024%2F06%2Fimage123.jpg&form-id=47&field-id=7&hash=0afa966d17976e112816ada2cc6b9275#1000x0 Proto:HTTP/1.1 ProtoMajor:1 ProtoMinor:1 Header:map[Accept:[image/*] User-Agent:[willnorris/imageproxy]] Body:<nil> GetBody:<nil> ContentLength:0 TransferEncoding:[] Close:false Host:REDACTED Form:map[] PostForm:map[] MultipartForm:<nil> Trailer:map[] RemoteAddr: RequestURI: TLS:<nil> Cancel:<nil> Response:<nil> ctx:0xc000038020} (served from cache: false)
tanrax commented 2 months ago

Same problem with Redis connection

imageproxy -addr 0.0.0.0:8080 -cache redis://:@redis:6379/0
willnorris commented 2 months ago

what caching headers is the upstream server sending? It's been a little while, but there was some discussion some time ago about overriding upstream caching headers, but I don't think it was ever implemented: https://github.com/willnorris/imageproxy/issues/28

tanrax commented 2 months ago

Hi @willnorris 

        proxy_set_header Connection "upgrade";
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
        proxy_set_header X-Forwarded-Host $server_name;
        proxy_set_header X-Real-IP $remote_addr;

However, the thumbnails are not in the folder. The CPU is always running. Imageproxy performs the resize but never saves the output.

willnorris commented 2 months ago

But what headers are sent from the server hosting the image? For example, the codercat image from GitHub specifies an etag, an expires timestamp, and a cache-control header:

% curl -I https://octodex.github.com/images/codercat.jpg
HTTP/2 200 
etag: "64f0b60d-70bf"
expires: Sun, 08 Sep 2024 16:54:00 GMT
cache-control: max-age=600

Those are what allow imageproxy to know how long it can cache the image. What happens if you try to load the codercat image in your proxy? Is it being served from cache? If so, then take a look at the response headers from the remote image you are loading to see if it has similar caching headers.