search

willnorris / imageproxy

A caching, resizing image proxy written in Go
Apache License 2.0
3.55k stars 483 forks source link

Persistent caching and no request on ressource after caching #352

Closed aymxdd closed 2 years ago

aymxdd commented 2 years ago

Hello guys,

I have a use case where the requested ressources needs to be called only once to save on bandwith.

Is it possible to add a flag to tell the proxy to only make one request to the image, then save it permanentelly on the s3 cache to be served exclusively from it the next time it's requested, without ever making another request to the ressource if it's already on cache ?

During my testings, I saw that the current behavior is that:

1) the proxy doesn't put the image on cache on the first call 2) It still makes a request to that URL anyways even if it's already cached

I'm really bad at golang and I would really appreciate if someone could help me on this !

willnorris commented 2 years ago

imageproxy should respect the cache headers on the remote request. So if the remote server doesn't provide any caching instructions (or explicitly says not to cache), then it likely won't get cached. But if it's getting cached and still refetching, that seems weird. It would be helpful to see what options you're running imageproxy with and the remote URL in question.

There's also been discussion about overriding the cache directives from the remote server, which we'll probably still end up adding at some point: https://github.com/willnorris/imageproxy/pull/208

aymxdd commented 2 years ago

Thanks for the quick response !

I'm only using the -cache flag followed by a s3 URL (that's working since my s3 bucket is filling up nonetheless) and the remote URL I used was a dummy node API that I built just for the test that doesn't include a cache header, that may be the reason it still makes a request on the source. But the production enpoints will be a pinata API used to fetch NFTs on IPFS.

Here's an example URL of what I'm requesting: https://craft-network.mypinata.cloud/ipfs/QmUmYz1AmGEkrwW9bGiqmjy375RR4YBCbAZxpiMtN3ZJmg

The override option from #208 could indeed be my savior !

aymxdd commented 2 years ago

Nevermind, I used a network analysis tool to see if the proxy makes a request to my source and it didn't ! It's all AWS and nothing goes to pinata, my bad and thanks for your time and for this good software !