Implement a new cache system natively compatible to Caddy

[darkweak]

Description

Hello there 👋,
I suggest to implement a new cache system called Souin in addition to the Varnish one.
The advantages would be that Souin can be used directly as caddy module which replace Nginx. I tried to implement it directly in the api-platform stack and the only one issue I am facing is the mercure and vulcain pushes. I don't know how to handle that, which routes can I exclude (for mercure I could exclude .well-known routes but vulcain is a little bit more tricky). I handle actually the route/resource invalidation with the bundle Souin API Platform bundle which one implements the PurgerInterface (note that this bundle is currently in a beta state / poc).
Souin handles your CDN provider invalidation (like Fastly, Akamaï, Cloudflare (next release)) and set the correct HTTP headers to set your cache inside your CDN too (and obviously store the response inside Souin).

Example

{
    order cache before rewrite
    # Debug
    {$DEBUG}
    # HTTP/3 support
    servers {
        protocol {
            experimental_http3
        }
    }
    cache {
        api {
            souin {
                enable true
            }
        }
        regex {
            exclude /test2.*
        }
        headers Content-Type Authorization
        log_level debug
        ttl 1000s
        stale 2000s
    }
}

#...

route {
    cache
    ... other configuration

# config/api_platform.yaml
api_platform:
        # ...
        http_cache:
                invalidation:
                    enabled: true
                public: true

# config/souin_api_platform.yaml
souin_api_platform: # minimal to enable the Souin bridge

[dunglas]

Hi!

It would be great to have native support for Souin. I fail to see what's the issue with Vulcain and Mercure. This shouldn't interfer with an HTTP cache server. Could you be a bit more specific?

[darkweak]

I don't know why this behaviour appear, but in some cases (e.g. when I try to use the swagger UI) it doesn't work but when I try to load the cache after mercure and vulcain it works. 🤷‍♂️

[darkweak]

Update, related to the encode zstd gzip directive

[darkweak]

Fixed in the https://github.com/darkweak/souin/pull/138 with the optimisations. I hope to merge the previous PR today and release in few days the fully working invalidation from the API Platform system. After that I will add the AddTagsListener and we'll be good.

[soyuka]

we implemented xkey/ykey in https://github.com/api-platform/core/pull/4340 I guess there's not much to do to be compatible with souin? (change the header name?)

[darkweak]

xkeys are deprecated and the ykeys are not very standard. Using the Surrogate-Key is more relevant.

[darkweak]

And I have to implement the AddTagsListener to be able to notify the probable CDN or other proxy to cache the resource by tag

[soyuka]

xkeys are deprecated and the ykeys are not very standard. Using the Surrogate-Key is more relevant.

the specification is almost the same, our implementation works all you need is to change the separator

[darkweak]

our implementation works all you need is to change the separator

Your implementation works, but I'm sorry to tell you that I won't use a deprecated adapter (Souin side) to support your Varnish integration. And you want to me that I use an implementation despite Varnish is not anymore in the production stack and it won't be available as a caddy module?

[dunglas]

The open source version is not deprecated as far as I can tell: https://github.com/varnish/varnish-modules/blob/master/src/vmod_xkey.vcc

This module is still actively maintained.

Only the proprietary version from Varnish Software (not the same as Varnish the FOSS project) is deprecated.

Anyway having the ability to configure the header name and the separator covers almost all formats.

[darkweak]

Anyway having the ability to configure the header name and the separator covers almost all formats.

Did I miss something?

[dunglas]

I mean that API Platform and Souin could provide a generic solution simply by providing configuration options to set the name of the header to use (xkey, ykey, Surrogate-Key or Cache-Tags for instance) and the separator (` or, `). This would allow to use both tools with almost all cache providers/services.

[darkweak]

IMHO API Platform should support this instead of Souin because there are some limitations in Varnish that are not in other proxies. We could work on it during the API Platform hackathon and provide a generic solution for the v3 I guess. Maybe I can find some time this week to introduce a POC about this abstraction and this generalisation.

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.