matthewwithanm / django-imagekit

Automated image processing for Django. Currently v4.0
http://django-imagekit.rtfd.org/
BSD 3-Clause "New" or "Revised" License
2.26k stars 276 forks source link

Too many cache/redis calls #438

Closed wsantos closed 6 months ago

wsantos commented 6 years ago

I've facing a problem where a have a page with a huge list of images and this is taking too long on redis (aprox.: 750 Redis calls), I was checking the code and this could be fixed checking if the file is on filesystem already instead of first relying on redis/cache first, do you this is a doable solution ? If you agree with it I can create a PR for it.

Here it's the line mentioned. https://github.com/matthewwithanm/django-imagekit/blob/7e233841457d088981991d331b2b849426f7b31c/imagekit/cachefiles/backends.py#L66

Regards.

vstoykov commented 6 years ago

The cache is there in order not to touch the file storage (it can be remote on S3 for example).

You can try to see how fast it will be if you completely disable imagekit cache:

CACHES = {
    'default': {
        # Your default cache config    
    },
    'dummy': {
        'BACKEND': 'django.core.cache.backends.dummy.DummyCache',
    }
}
IMAGEKIT_CACHE_BACKEND = 'dummy'

If this is also slow (because there will be too many syscalls to the filesystem), then you need to think of some page fragment caching on your page. Once the images are created and rendered the HTML probably will not change so often and there is no need to ask ImageKit to check if the file is generated for particular image.

wsantos commented 6 months ago

We've created a backend that check files in disk instead of redis since we don't use S3.

Thank you!