search

mmikkel / CacheFlag-Craft

Flag and clear template caches
MIT License
36 stars 5 forks source link

Orphaned caches #21

Closed rungta closed 6 years ago

rungta commented 6 years ago

We’re using this plugin on a heavy traffic website where content is added multiple times a day. This means: caches are cleared fairly frequently, and caches are regenerated rapidly.

I’ve noticed a rare bug where at times a cache that should have been cleared because one of its flag was invalidated continues to persist. Even manually clearing the relevant flag’s cache from the control panel does not clear the cache. I was mystified the first time this happened, but after digging around the DB, I discovered that certain CacheFlag generated records in the templatecaches table had been orphaned and had no associated records in the templatecaches_flagged table.

Basically, the result of this query should have been zero rows (assuming no non-CacheFlag generated cache records) but it wasn’t an empty set:

SELECT `id`, `cacheKey`, `expiryDate`, `path` FROM `craft_templatecaches` WHERE `id` NOT IN (SELECT `cacheId` FROM `craft_templatecaches_flagged`);

I went ahead and manually deleted the orphaned records from the table. Everything was working smoothly for a while. But then this issue resurfaced, and sure enough, there were some orphaned caches once again.

I am now starting to suspect that either the cache deletion or creation queries aren’t being executed as a single transaction, and that is somehow leading to certain templatecaches records being either generated without corresponding templatecaches_flagged records, or the templatecaches_flagged records are being deleted without the actual templatecaches record being deleted.

Any help or insight into this would be much appreciated. Thanks!

mmikkel commented 6 years ago

Hi @rungta, very sorry for the extremely late reply – but better late than never, right?

I think your theory about the the templatecaches being generated or deleted without appropriate action to the templatecaches_flagged table is probably correct, due to how the plugin is architectured. I haven't seen an issue like this myself, but I also have never battle-tested the plugin on a very high-traffic site (additionally, I generally use pretty short TTL times for my caches, so even if the issue has occurred I might not've noticed).

To be honest, I can't promise that I'll be able to resolve this issue for the Craft 2 version of CacheFlag. When the Craft 3 version happens, I have some ideas about how to do things differently, which should negate the issue entirely. If feasible, I'll backport stuff over to the Craft 2 version, but again – no promises :)

mmikkel commented 6 years ago

Hi again @rungta – Cache Flag is available for Craft 3 now, check out https://github.com/mmikkel/CacheFlag-Craft3.

I took a different approach to cache flagging with the Craft 3 version, so the orphaned caches thing shouldn't happen anymore. It's not really practical to backport this to the Craft 2 version, though, sorry.

rungta commented 6 years ago

Better late than never, for sure! I understand backporting might be difficult, but hoping for the best. We’ve created a cron job to delete orphaned caches from the database directly as a stopgap measure.

rungta commented 6 years ago

Didn’t see your last comment before posting. Yay for the Craft 3 release and good to know this has been addressed there.