redis2_query set fails with timeout

[pmezykowski]

Hi, I think I hit the limitation of the library. Whenever I try to store the response in a size aprox. 120 000 characters and more, I get the timeout. The library is used in a srcache configuration, having the store endpoint configured as follow:

     location /Lorem/ {
        set $key "cache:$uri:$http_X_Custom_Cache_Vary_By";
        set_escape_uri $escaped_key $key;

        srcache_request_cache_control off;
        srcache_response_cache_control off;
        srcache_ignore_content_encoding on;
        srcache_store_statuses 200;

        srcache_fetch GET /redis $key;
        srcache_store PUT /redis2 key=$escaped_key&exptime=600; # 10x60s = 600s

        add_header X-Fetch-Status $srcache_fetch_status;

        proxy_pass https://host.docker.internal:7193/lorem/;

        add_header X-Store-Status $srcache_store_status;
    }

    location = /redis2 {
        internal;

        set_unescape_uri $exptime $arg_exptime;
        set_unescape_uri $key $arg_key;

        redis2_query auth "${REDIS_PASSWORD}";
        redis2_query set $key $echo_request_body;
        redis2_query expire $key $exptime;
        redis2_pass "${REDIS_ADDRESS}";
    }

I created a dummy webservice to proxy to, that responds with the text of the length passed as parameter. I also launched redis-cli monitor to see the traffic, and turns out that:

• the response with a text length of 110 000 characters is being stored in a cache without problem
• the response with a text length of 120 000 characters is timing out exactly on redis2_query set $key $echo_request_body;
  • in redis-cli monitor I see the auth "${REDIS_PASSWORD}" command being executed):
  • in nginx logs I see two entries that appear with a delay:
  • [error] 10#10: *10 upstream timed out (110: Operation timed out) while reading upstream, client: 172.17.0.1, server: localhost, request: "GET /Lorem/120000 HTTP/1.1", subrequest: "/redis2", upstream: "redis2://192.168.65.2:6379", host: "localhost:9000"
  • [error] 10#10: *10 srcache_store subrequest failed: rc=0 status=504 while sending to client, client: 172.17.0.1, server: localhost, request: "GET /Lorem/120000 HTTP/1.1", subrequest: "/redis2", upstream: "redis2://192.168.65.2:6379", host: "localhost:9000"
• I also doubt it is a problem with $echo_request_body, because there was no problem with following code:

  set $temp $echo_request_body; 
  redis2_query set $key "Test123"; 

[gridpane]

Having the same issue...

Page has a form that is returning an overly large JSON and its causing timeouts, page doesn't cache, redis times out after 1m and then the page/form loads...

2024/07/22 18:43:36 [error] 1160421#1160421: *89852 upstream timed out (110: Connection timed out) while reading upstream, client: XXXXXX server: XXXXX, request: "GET / HTTP/2.0", subrequest: "/redis-store", upstream: "redis2://unix:/var/run/redis-page/redis-page.sock:", host: "XXXXX"
2024/07/22 18:43:36 [error] 1160421#1160421: *89852 srcache_store subrequest failed: rc=0 status=504 while sending to client, client: XXXXX, server: XXXXX, request: "GET / HTTP/2.0", subrequest: "/redis-store", upstream: "redis2://unix:/var/run/redis-page/redis-page.sock", host: "XXXXX"