search

mikebronner / laravel-model-caching

Eloquent model-caching made easy.
MIT License
2.26k stars 217 forks source link

TTL Setting #261

Closed aerjotl closed 5 years ago

aerjotl commented 5 years ago

Hi guys,

I just launched new Laravel version of my website using your library. It works great so far, but I had to double the amount of RAM of my server, because Redis cache started growing really fast (more then 2GB of memory usage).

I was wondering if it would be easy to implement a TTL Setting, that could be used instead of rememberForever. Are there any reasons why it would not work? I can propose a PR, but I would like to ask your opinion first.

I saw that similar issue was already discussed in the past https://github.com/GeneaLabs/laravel-model-caching/issues/72, but for some reason it was rejected.

thanks.

mikebronner commented 5 years ago

Hi @aerjotl, thanks for reaching out and explaining how you use this package! :)

The reason I have not entertained adding a TTL option, is because Redis and Memcached self-manage any stale records, as well as memory usage.

You say you "had to" double the amount of ram on the server. What prompted you to do so? Where you getting errors? For production environments, I do not recommend having in-memory Redis server running on the same server as the web app or database, but instead spin up a dedicated Redis server.

Does this explanation help? I would love to learn more about the memory requirements you mentioned, and what prompted you to increase server memory. Thanks!

aerjotl commented 5 years ago

Hi @mikebronner, maybe I was not clear in explaining my issue. The old version of the website did not use Redis. We stored the cache in files, but we had a TTL setting, which was very helpful in managing cache size and overall performance. We have a website with more than 100.000 of song lyrics. Some of them are very popular, and some of them are not. If we put all of them in cache they would consume a lot of RAM. Instead, we would like to keep only the popular ones, and expire the ones that are not accessed so often.

Redis manages stale records, but, as far as I know, it does it based on key TTL. I'm not sure what's the strategy if the expiration of all the keys is set to 'never'.

mikebronner commented 5 years ago

@aerjotl Thanks for explaining your situation some more. I will go back and read up on Redis and see what I can find out about cache expiring methods, and seeing if this can't be addressed there.

aerjotl commented 5 years ago

hi @mikebronner, we managed to tweak Redis configuration and indeed, it's quite smart and manages keys without TTL properly.

However, I have one use case for which a TTL would be helpful: We have a model TopViews, that represents the most popular items on the website. The backing table is updated by a perl script, based on the log files. We need to clear the cache for that model regularly, to make sure the changes are reflected on the website. An option of TTL for this particular model would solve the problem. What do you think?

mikebronner commented 5 years ago

@aerjotl Glad you were able to sort out the Redis configuration! :) I don't believe TTL is the right approach for your other issue, as the cache would not necessarily invalidate when your Perl scripts makes the updates. Rather, what you should do is have the Perl script also invalidate the model(s) by using the artisan command:

php artisan modelCache:clear --model=App\\MyModel

That would be the best way.

mikebronner commented 5 years ago

I'll close this issue, as you seem to have things working.

zero0cool0 commented 6 months ago

For anyone else coming across this issue and wondering how to tweak the Redis configuration as mentioned above, read this.

Essentially, the configuration statements maxmemory and maxmemory-policy allow you to set an upper memory limit and define an eviction policy in case maxmemory is reached, which in my case I set to allkeys-lfu, meaning that the least frequently used keys get evicted first.