uri-miss on basic traefik middleware config

[tfyl]

I was trying to get a basic version of souin working on a traefik config (running from a k8s CRD config)

apiVersion: traefik.containo.us/v1alpha1
kind: Middleware
metadata:
  name: http-cache
  namespace: services
spec:
  plugin:
    souin:
      default_cache:
        ttl: 120s
      log_level: debug

also used the aforementioned middleware on a specified route and enabled the plug-in using the args:

   --experimental.plugins.souin.modulename=github.com/darkweak/souin
   --experimental.plugins.souin.version=v1.6.22

I can see that the souin middleware is being called due to Cache-Status: Souin; fwd=uri-miss being included in the response header. However after the initial GET request, my understanding is they should be cached from then on (for 120s), instead, requests seem to continue hitting the origin server (container) and return Cache-Status: Souin; fwd=uri-miss.

I have attempted adding other optional parameters but to no avail. Is there something I am missing in the config which is causing the uri-miss as I cannot seem to get any response which is retrieved from cache?

Thank you in advance.

[darkweak]

Hello @tfyl can you paste the request with the headers and the response with the headers too?

[VladyslavLukyanenko]

Hello @tfyl can you paste the request with the headers and the response with the headers too? I'll reply on Tfyl's behalf since we work both at the same company.

Request URI: https://endpoint.io/api/metadata/0A005C84F8FB9271

Request Headers: Authorization: Bearer (JWT TOKEN)

Response Headers

HTTP/2 200 access-control-allow-origin: * cache-status: Souin; fwd=uri-miss content-type: application/json date: Thu, 20 Oct 2022 09:20:01 GMT server: istio-envoy vary: Origin x-envoy-upstream-service-time: 17 x-request-id: ec42f585-9c4c-9cbe-91b8-283e5de5975a content-length: 289

[darkweak]

Hello @VladyslavLukyanenko Can you try without a k8s environment? What's your træfik version?

That's weird because on the E2E tests everything is working, the headers are present and the response is stored.

[VladyslavLukyanenko]

Hello @VladyslavLukyanenko Can you try without a k8s environment? What's your træfik version?

That's weird because on the E2E tests everything is working, the headers are present and the response is stored.

Traffic version is 2.9.1 Well the use case we want to use is inside k8s environment.

[darkweak]

That's just to check if that's a Træfik version issue, k8s one or in the cache middleware itself. Because of they use Yægi as interpreter and they make some BC without notices we have to try these use cases. Anyway, I'll investigate on that and keep you updated.

[darkweak]

I cannot reproduce on a docker-compose stack

version: '3'

services:
  traefik:
    image: traefik:v2.9.1
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
    environment:
      GOPATH: /plugins-local
    command:
      - --api.dashboard=true
      - --api.insecure=true
      - --experimental.plugins.souin.modulename=github.com/darkweak/souin
      - --experimental.plugins.souin.version=v1.6.22
      - --providers.docker=true
      - --entrypoints.https.address=:443
    ports:
      - 80:80
      - 443:443
      - 8080:8080

  whoami:
    image: traefik/whoami
    labels:
      - traefik.http.routers.whoami.rule=Host(`domain.com`)
      - traefik.http.routers.whoami.entrypoints=https
      - traefik.http.routers.whoami.tls=true
      - traefik.http.routers.whoami.middlewares=souin
      - traefik.http.middlewares.souin.plugin.souin.default_cache.ttl=120s
      - traefik.http.middlewares.souin.plugin.souin.log_level=debug

Can you check that you don't add a Cache-Control: no-cache header?

[VladyslavLukyanenko]

I cannot reproduce on a docker-compose stack

version: '3'

services:

  traefik:

    image: traefik:v2.9.1

    volumes:

      - /var/run/docker.sock:/var/run/docker.sock

    environment:

      GOPATH: /plugins-local

    command:

      - --api.dashboard=true

      - --api.insecure=true

      - --experimental.plugins.souin.modulename=github.com/darkweak/souin

      - --experimental.plugins.souin.version=v1.6.22

      - --providers.docker=true

      - --entrypoints.https.address=:443

    ports:

      - 80:80

      - 443:443

      - 8080:8080

  whoami:

    image: traefik/whoami

    labels:

      - traefik.http.routers.whoami.rule=Host(`domain.com`)

      - traefik.http.routers.whoami.entrypoints=https

      - traefik.http.routers.whoami.tls=true

      - traefik.http.routers.whoami.middlewares=souin

      - traefik.http.middlewares.souin.plugin.souin.default_cache.ttl=120s

      - traefik.http.middlewares.souin.plugin.souin.log_level=debug

Can you check that you don't add a Cache-Control: no-cache header?

I don't include that header

[darkweak]

Either you have a request header Cache-Control: no-cache or another middleware that add this header. I don't know why. you run into this issue and I'm not able to reproduce.

[VladyslavLukyanenko]

Either you have a request header Cache-Control: no-cache or another middleware that add this header. I don't know why. you run into this issue and I'm not able to reproduce.

What headers are needed for cache to work? Also does the Redis provider still work? I tried to make it work since http://dragonflydb.io/ works with Redis protocol but no luck

[darkweak]

You should not send a Cache-Control: no-cache HTTP request header. Redis is not supported yet inside the træfik middleware.

[VladyslavLukyanenko]

You should not send a Cache-Control: no-cache HTTP request header. Redis is not supported yet inside the træfik middleware.

Great, is there a way I could contribute to add Redis into traefik middleware?

[VladyslavLukyanenko]

Also we discovered why it doesn't cache, if we include Authorisation header it returns cache-status: Souin; fwd=uri-miss but if we disable the header it does cache the response

[darkweak]

Yep you could add that and I'll appreciate that. You'll have to copy/paste the redis provider from https://github.com/darkweak/souin/blob/master/cache/providers/redisProvider.go in the plugins/traefik/override/providers folder. Update the plugins/traefik/main.go and add the case about redis and deserialize the configuration in it (it should match the provider configuration, e.g. https://github.com/darkweak/souin/blob/master/plugins/roadrunner/configuration.go#L181). Update the InitializeProvider function located in the plugins/traefik/override/providers/abstractProvider.go, add a condition, if the configuration object has a redis configuration (configuration.GetDefaultCache().GetRedis() != nil) and return either the RedisConnectionFactory, or the CacheConnectionFactory otherwise. After that you'll have to run the make vendor-plugins command and everything should be ok.

Oh I missed the Authorization HTTP header in the request, my bad. An authenticated request must not be stored because that's a shared cache (shared between multiple users).

[VladyslavLukyanenko]

Yep you could add that and I'll appreciate that. You'll have to copy/paste the redis provider from https://github.com/darkweak/souin/blob/master/cache/providers/redisProvider.go in the plugins/traefik/override/providers folder. Update the plugins/traefik/main.go and add the case about redis and deserialize the configuration in it (it should match the provider configuration, e.g. https://github.com/darkweak/souin/blob/master/plugins/roadrunner/configuration.go#L181). Update the InitializeProvider function located in the plugins/traefik/override/providers/abstractProvider.go, add a condition, if the configuration object has a redis configuration (configuration.GetDefaultCache().GetRedis() != nil) and return either the RedisConnectionFactory, or the CacheConnectionFactory otherwise. After that you'll have to run the make vendor-plugins command and everything should be ok.

Oh I missed the Authorization HTTP header in the request, my bad. An authenticated request must not be stored because that's a shared cache (shared between multiple users).

is there a way to forcefully cache it with Souin or shall I look for another tool for this use-case?

[darkweak]

The RFC is clear about that.

A shared cache MUST NOT use a cached response to a request with an Authorization header field (Section 4.2 of [RFC7235]) to satisfy any subsequent request unless a cache directive that allows such responses to be stored is present in the response.

You can use another HTTP cache proxy BUT if it supports well the RFC then the request won't be stored.

[tfyl]

Thank you for helping getting this resolved