search

colinmollenhour / Cm_Cache_Backend_Redis

A Zend_Cache backend for Redis with full support for tags (works great with Magento)
Other
389 stars 142 forks source link

Added chunking logic to the SUNION call to avoid lengthy blocking. #89

Closed davidalger closed 9 years ago

davidalger commented 9 years ago

Added support to getIdsMatchingAnyTags for chunking the SUNION calls to mitigate long-running SUNION calls from blocking redis connection requests. The chunk size defaults to 500 but is configurable via the sunion_chunk_size setting in the backend_options of the redis cache configuration.

davidalger commented 9 years ago

Note: Being unfamiliar with using the Lua scripting, I did not attempt to alter that to use the new configuration option for chunking when in Lua mode.

colinmollenhour commented 9 years ago

Thanks, David!

sksonic commented 7 years ago

@colinmollenhour We have experienced the same issue with the SUNION call blocking redis. The chunking option above has helped us but we had to disable lua scripting. Can a similar chunking logic be applied when lua scripting is enabled?

colinmollenhour commented 7 years ago

It probably could be, but unfortunately this is not something I have the time to tackle. However, I would suggest investigating to see why you have so many tags; to me having to query for ids matching over 500 tags indicates an issue with improper tagging strategy.