darkweak / souin

An HTTP cache system, RFC compliant, compatible with @tyktechnologies, @traefik, @caddyserver, @go-chi, @bnkamalesh, @beego, @devfeel, @labstack, @gofiber, @go-goyave, @go-kratos, @gin-gonic, @roadrunner-server, @zalando, @zeromicro, @nginx and @apache
https://docs.souin.io
MIT License
719 stars 56 forks source link

Surrogate Keys not working with otter #563

Closed mac-chaffee closed 1 month ago

mac-chaffee commented 1 month ago

Hello! I'm trying to understand how to use surrogate keys and can't seem to get it to work by merely following the spec.

My use-case is that I need to associate a group of cached files with a "project ID" so I can purge all files for the project when it is modified. I'm using Caddy and Otter.

Here's my Caddy config (using github.com/darkweak/souin/plugins/caddy@v1.7.2)

{
    cache {
        ttl 300s
        max_cacheable_body_bytes 1000000
        otter
        api {
            souin
        }
        # This may be required to enable surrogate-keys? https://github.com/darkweak/souin/issues/306#issuecomment-1429744951
        cdn {
            dynamic
            provider souin
        }
        default_cache_control public
    }
}

localhost {
    route {
        cache
        reverse_proxy web:3000
    }
}

When I want to set a surrogate-key, I include the header in the response in my application code (web:3000):

$ curl -ik -H "Host: picouser-www.localhost" https://localhost                        
HTTP/2 200 
alt-svc: h3=":443"; ma=2592000
cache-control: public
cache-status: Souin; fwd=uri-miss; stored; key=GET-https-picouser-www.localhost-/
content-security-policy: script-src 'self'
content-type: text/html
date: Tue, 22 Oct 2024 02:16:29 GMT
etag: 5fddc78468241bf0dc7e721867874288
last-modified: Mon, 21 Oct 2024 20:21:37 GMT
server: Caddy
strict-transport-security: max-age=31536000; preload
surrogate-key: picouser-www                                <------ Project ID surrogate key is set here
x-frame-options: DENY
content-length: 2051

But the surrogate key is never saved:

$ curl 'https://localhost/souin-api/souin/surrogate_keys' | jq
{
  "GET-https-picouser-www.localhost-/": "<omitted>",
  "IDX_GET-https-picouser-www.localhost-/": "\n�\u0001\n\"GET-https-picouser-www.localhost-/\u0012k\n\u000b\b��ܸ\u0006\u0010���Y\u0012\n\b��ܸ\u0006\u0010��\u0004\u001a\n\b��ܸ\u0006\u0010��\u0004* 5fddc78468241bf0dc7e7218678742882\"GET-https-picouser-www.localhost-/"
}

Likewise, purging via the surrogate key I set also does not clear this cache entry.

I searched through the code and is seems like what I'm doing should work, but I may be missing a config value or something: https://github.com/darkweak/souin/blob/2968d51eef04b31925352ac4aaf815ea49f4efbd/pkg/surrogate/providers/common.go#L238

Am I using the feature incorrectly? Or could there be a bug? Happy to provide further info!

darkweak commented 1 month ago

Hello @mac-chaffee the change https://github.com/darkweak/storages/pull/17/files#diff-f6400dfc406b7de4482980fec3748055e9bb49739340994735b5ed509a1e49c7L67-R67 should patch your issue you can try it with --with github.com/darkweak/storages/otter/caddy@f6400dfc406b7de4482980fec3748055e9bb49739340994735b5ed509a1e49c7L67 --with github.com/darkweak/storages/otter@f6400dfc406b7de4482980fec3748055e9bb49739340994735b5ed509a1e49c7L67. Btw that will be so great if picosh uses Souin/cache-handler module.