openresty / srcache-nginx-module

Transparent subrequest-based caching layout for arbitrary nginx locations.
http://wiki.nginx.org/NginxHttpSRCacheModule
476 stars 105 forks source link

$srcache_store_status always shows BYPASS #74

Closed mbtamuli closed 5 years ago

mbtamuli commented 6 years ago

This is the config I am using

set $skip 0;
if ($request_method = POST) {
  set $skip 1;
}
if ($query_string != "") {
  set $skip 1;
}

if ($request_uri ~* "(/wp-admin/|/xmlrpc.php|wp-.*.php|index.php|/feed/|sitemap(_index)?.xml|[a-z0-9_-]+-sitemap([0-9]+)?.xml)") {
  set $skip 1;
}

if ($http_cookie ~* "comment_author|wordpress_[a-f0-9]+|wp-postpass|wordpress_no_cache|wordpress_logged_in|woocommerce_items_in_cart") {
  set $skip 1;
}

location / {
  try_files $uri $uri/ /index.php?$args;
}

location /redis-fetch {
  internal  ;
  set  $redis_key $args;
  redis_pass  redis:6379;
}

location /redis-store {
  internal  ;
  set_unescape_uri $key $arg_key ;
  redis2_query  set $key $echo_request_body;
  redis2_query expire $key 14400;
  redis2_pass  redis:6379;
}

location ~ \.php$ {
  set $key "nginx-cache:$scheme$request_method$host$request_uri";
  try_files $uri =404;

  srcache_fetch_skip $skip;
  srcache_store_skip $skip;

  srcache_request_cache_control off;

  set_escape_uri $escaped_key $key;

  srcache_fetch GET /redis-fetch $key;
  srcache_store PUT /redis-store key=$escaped_key;

  more_set_headers 'X-SRCache-Fetch-Status $srcache_fetch_status' 'X-SRCache-Store-Status $srcache_store_status';

  include fastcgi_params;
  fastcgi_pass php:9000;
}
curl -IL http://example.site/hello-world/
HTTP/1.1 200 OK
Server: openresty
Date: Wed, 29 Aug 2018 09:40:49 GMT
Content-Type: text/html; charset=UTF-8
Connection: keep-alive
Vary: Accept-Encoding
X-Powered-By: PHP/7.2.8
X-Pingback: http://example.site/xmlrpc.php
Link: <http://example.site/wp-json/>; rel="https://api.w.org/"
Link: <http://example.site/?p=1>; rel=shortlink
X-SRCache-Fetch-Status: MISS
X-SRCache-Store-Status: BYPASS

The $srcache_fetch_status is correctly populated with MISS or HIT. But $srcache_store_status always reporst BYPASS.

I've also tried adding the following directives before srcache_fetch and srcache_store with no difference to the output.

  srcache_response_cache_control off;
  srcache_ignore_content_encoding on;
  srcache_methods GET HEAD;

  srcache_store_hide_header X-SRCache-Fetch-Status;
  srcache_store_hide_header X-SRCache-Store-Status;

I've also attached the error log(with debugging enabled) - error.log

nhanledev commented 6 years ago

I have the same issue when debugging Redis Cache at Nginx Level by using srcache module.

The document about this cache status was fully written at here. It only STORE the HTTP 200, 301, and 302 result by default..

I write this comment just for reference, to help anybody go after me understand that this is not a problem.

Cited from the above link

srcache_store_statuses
syntax: srcache_store_statuses <status1> <status2> ..

default: srcache_store_statuses 200 301 302

context: http, server, location, location if

phase: output-header-filter

This directive controls what responses to store to the cache according to their status code.

By default, only 200, 301, and 302 responses will be stored to cache and any other responses will skip srcache_store.

You can specify arbitrary positive numbers for the response status code that you'd like to cache, even including error code like 404 and 503. For example:

 srcache_store_statuses 200 201 301 302 404 503;
At least one argument should be given to this directive.

This directive was first introduced in the v0.13rc2 release.
mbtamuli commented 5 years ago

@nhanledev You are correct in that by default only responses with 200, 301 and 302 status code will be stored.

This directive controls what responses to store to the cache according to their status code.

But my whole question was regarding this variable getting wrong value after the response has been stored. srcache_store_status

The $srcache_fetch_status is correctly populated with MISS or HIT. But $srcache_store_status always reports BYPASS. This srcache_store_status variable should show STORE on the request getting correctly stored.

agentzh commented 5 years ago

@mbtamuli It sounds like you are reading the $srcache_store_status variable value too early, that is, before the srcache_store's output filter is run. Check the relative running order between your srcache's output filter and the ngx_headers_more module's output filter. It seems that you get the order wrong.

mbtamuli commented 5 years ago

@agentzh Okay. I have already attached my configuration above.

  srcache_store PUT /redis-store key=$escaped_key;

  more_set_headers 'X-SRCache-Fetch-Status $srcache_fetch_status' 'X-SRCache-Store-Status $srcache_store_status';

I believed this order mattered. Can you help me to understand how to

Check the relative running order between your srcache's output filter and the ngx_headers_more module's output filter

In the log(that I attached in the first comment), I found

2018/08/29 04:58:43 [debug] 9#9: *1 http fastcgi parser: 1
2018/08/29 04:58:43 [debug] 9#9: *1 http fastcgi header done
2018/08/29 04:58:43 [debug] 9#9: *1 headers more header filter, uri "/index.php"
2018/08/29 04:58:43 [debug] 9#9: *1 http script var: "MISS"
2018/08/29 04:58:43 [debug] 9#9: *1 http script copy: "<wierd character>"
2018/08/29 04:58:43 [debug] 9#9: *1 http script var: "BYPASS"
2018/08/29 04:58:43 [debug] 9#9: *1 http script copy: "<wierd character>"
2018/08/29 04:58:43 [debug] 9#9: *1 http script var: "0"
2018/08/29 04:58:43 [debug] 9#9: *1 srcache_store decides to store the response
2018/08/29 04:58:43 [debug] 9#9: *1 posix_memalign: 0000555B8CAFDAE0:4096 @16
2018/08/29 04:58:43 [debug] 9#9: *1 srcache store header HTTP/1.1 200 OK

There was a character here that could not be copied.

Are you talking about this? The fact that headers more header filter, uri "/index.php" ran before srcache store header HTTP/1.1 200 OK?

How can I modify my config to fix that?

agentzh commented 5 years ago

@mbtamuli When in doubt, you can just swap the order of the --add-module=PATH option of your ./configure command line when building your nginx or openresty (or changing the add_module directives' order in your nginx.conf if you are using dynamic modules).