search

FRiCKLE / ngx_cache_purge

nginx module which adds ability to purge content from FastCGI, proxy, SCGI and uWSGI caches.
http://labs.frickle.com/nginx_ngx_cache_purge/
Other
1.07k stars 313 forks source link

After purging a specific variant, that variant may still be present in the cache #55

Open sharpobject opened 7 years ago

sharpobject commented 7 years ago

If you are using Vary, you might purge a specific variant, but that variant may still be present in the cache and served to clients.

In this example, the following sequence of requests is made for the Waifu: Mokou variant:

GET (MISS)
GET (HIT)
PURGE (404)
GET (HIT)

In this example, the following sequence of requests is made for the Waifu: Mokou variant:

GET (MISS)
GET (HIT)
GET (MISS)
PURGE (200)
GET (HIT)

Both of these tests "pass" over here on OSX 10.12.1 with nginx 1.9.15 and ngx_cache_purge 2.3.

taosx commented 7 years ago

Exactly same problem. I just found https://github.com/nginx-modules/ngx_cache_purge, maybe it's fixed.

sharpobject commented 7 years ago

@taosx My impression is that this issue is pretty fundamental to the way nginx's cache's Vary support works. So I would be surprised and impressed if someone fixed it from the module side.

I guess I should take this to the nginx mailing list. The issue is much less severe in the absence of ngx_cache_purge though, because the worst that can happen is like... you can MISS for some variant when you should HIT after some other variant gets deleted by LRU. Then you will store 2 copies of the same variant for a while.

sharpobject commented 7 years ago

Ok, I had a look and you can definitely successfully purge all variants of a resource if you use ngx_walk_tree. I think this will be linear in the number of files in your entire cache. I don't really want that.

99th edit: The fork linked by taosx above will have the same behavior for purging a single file, but will purge all variants if you use partial key purge.

ennorehling commented 3 years ago

I investigated this, and decided that just as @sharpobject says, ngx_walk_tree will look at every file in the cache, and worse, the plugin will (in ngx_http_purge_file_cache_delete_partial_file) read each of those files to find the cache_key stored inside to decide if it matches the partial key.

Worse, it will then delete the file from disk, but if you have another layer of cache, like the file_open cache, the cache entry will not be invalidated, so it's possible that it can still be served from memory. You'd need to set fcn->exists = 0; for the relevant ngx_http_file_cache_node_t *fcn, I guess, but how do you find that?

Disclaimer: This is the first time I've looked at NGINX sources. Don't trust anything I speculate about.