Memory size exhausted by deleting by Tags

[cokuna-pavelkosolapov]

What type of issue is this?

Exception/Error/Warning/Notice/Deprecation

Operating system + version

Various

PHP version

php 8.2

Connector/Database version (if applicable)

File,Redis

Phpfastcache version

9.1.3 ✅

Describe the issue you're facing

When i try to delete cache by tags i get the error message "Allowed memory size of XXX bytes exhausted". There is a large amount of Cache data (more than 6k+ Files/Redis keys, 150MB+).

Expected behavior

Deleting of cache should work fine with any amount of data.

Code sample (optional)

No response

Suggestion to fix the issue (optional)

it seems data wasn't removed from memory while removing from file system/redis db.

References (optional)

No response

Do you have anything more you want to share? (optional)

No response

Have you searched in our Wiki before posting ?

• [X] I have searched over the Wiki

[github-actions[bot]]

Hello curious contributor ! Since it seems to be your first contribution, make sure that you've been:

• Reading and searching out our WIKI
• Reading and agreed with our Code Of Conduct
• Reading and understood our Coding Guideline
• Reading our README If everything looks unclear to you, tell us what 😄 The Phpfastcache Team

[Geolim4]

Hello,

You are simply managing too many cache items through the tags.

Tags are internally php-handled, so if you have memory issue, you have either too low php memory allocated or extremely high volume of cache items used through tags API.

Cheers, Georges

[Geolim4]

You can also try to deactivate useStaticItemCaching option which will disable internal static caching. But each request will query the backend again.

[cokuna-pavelkosolapov]

You are simply managing too many cache items through the tags.

yes, i know, i mentioned it.

You can also try to deactivate useStaticItemCaching option

Thanks, the useStaticItemCaching => false option helped. although it seems, the system works much slower with this option, better now without cache at all, lol

But each request will query the backend again.

I don't understand what you mean. Every time I want to get some key, will it be read from the cache and not remain in RAM for the next get while same run?

But why am I running out of memory when useStaticItemCaching => true? Each cache item is 10–50 bytes. Even when each cache item is loaded and deleted one by one, its total memory size should not exceed 128 MB.

[Geolim4]

Thanks, the useStaticItemCaching => false option helped. although it seems, the system work much slower with this option, better now without cache at all, lol

That's the purpose of internal static caching.

I don't understand what you mean. Every time I want to get some key, will it be read from the cache and not remain in RAM for the next get while same run?

When you call any getItem* method they're fetched from the driver once and stored in itemInstances for future reuse if useStaticItemCaching is not set to false. So when you query that cache item again the item object instance stay the same and is served again when you need it. https://github.com/PHPSocialNetwork/phpfastcache/blob/c82647aab9d7785eeb1414667649c091f3687eb1/lib/Phpfastcache/Core/Pool/CacheItemPoolTrait.php#L58

When useStaticItemCaching is set to false no static storage is used and the backend driver is queried each time you call getItem*` methods, but at a cost of being slower indeed.

This is a choice I've made when the PSR-6 specification came out. The behavior of static caching was unclear about how the developer implement the PSR-6 and if static storage was permitted or not: https://www.php-fig.org/psr/psr-6/#cacheitempoolinterface

The primary purpose of Cache\CacheItemPoolInterface is to accept a key from the Calling Library and return the associated Cache\CacheItemInterface object. It is also the primary point of interaction with the entire cache collection. All configuration and initialization of the Pool is left up to an Implementing Library.

[Geolim4]

Please note that instead of disabling useStaticItemCaching you can periodically call detachItem($item) or detachAllItems(). They will be removed from the static cache of Phpfastcache without being removed from the backend driver.

Allowing you to free the memory when you need it. It is a sort of garbage collector.

[cokuna-pavelkosolapov]

I looked in code and found the problem. \Phpfastcache\Core\Pool\TaggableCacheItemPoolTrait::deleteItemsByTags calls \Phpfastcache\Core\Pool\TaggableCacheItemPoolTrait::getItemsByTags and then \Phpfastcache\Core\Pool\TaggableCacheItemPoolTrait::fetchItemsByTagFromBackend, which receive all cache items by tags, which is not at all necessary while deleting.

This setup generally works well, but a problem causes when the Administrator modifies the common settings. In such cases, all cached data for each user needs to be cleared at once.

However, the issue causes with phpFastCache, which reads all cache data into memory before deletion, leading to memory overuse.

Maybe using of generators will solve this issue, but it may need to rebuild architecture a bit.

[Geolim4]

which receive all cache items by tags, which is not at all necessary while deleting.

That's where you wrong: when you delete one or more items, all related tags items must be fetched to get their index updated as well as their metatadata (expiration date) or get also deleted if they don't contain anymore cache keys. If I don't fetch the tags, then the tags item will get out of date and keep references to deleted cache items.

Maybe using of generators will solve this issue, but it may need to rebuild architecture a bit.

This would requires that every backend I implemented support "cursor" operations, which is not the case actually. Only few of them support that.

[cokuna-pavelkosolapov]

Well, then there is only one option left: to implement a special data extraction specifically for the deletion operation. Otherwise, solving the issue of overflowing memory is impossible. However, behind this problem, there will likely come the next one - the script execution time limit.

In my case, I’ll have to rethink either the way information is stored or seek an alternative solution for data caching.

Thank you for your assistance.

[Geolim4]

Well, then there is only one option left: to implement a special data extraction specifically for the deletion operation. Otherwise, solving the issue of overflowing memory is impossible. However, behind this problem, there will likely come the next one - the script execution time limit.

Again, this is not supported by all drivers. Tags should be limited to a reasonable use and can't fit on any implementations cases due to abstraction limitations. Keep in mind that, if I implement something it must be at least supported by the majority of backends implemented.

V9.2 will be shipped next month with a major performance improvement for people fetching multiple items: https://github.com/PHPSocialNetwork/phpfastcache/wiki/%5BV5%CB%96%5D-Fetching-all-keys#as-of-92-january-2024

And even there, I'm limited by some backends to permitting me to grab multiple keys in a single calls. So it will automatically fallback on the current "loop key-by-key" implementation.

The good thing of supporting multiple backend is that it allows you to implements once the PSR6 code and switch quickly of backend when you need it. The bad thing is that it require an abstract code that fit for all those backends.