How to cleanup cache backend and remove generated thumbnails from storage

[hellysmile]

I can not find propper way how to cleanup cache and storage

Model.image.delete()
# or
Model.delete()

doesnt delete ImageSpecField data related to this model\field and ImageSpecField havent methods like delete() for manual cleanup

[matthewwithanm]

Just to reiterate (and maybe expand on) what I said in IRC for other users' benefit:

There's currently not a great way to do this. We removed the feature for it in #221 because we figured it was better to not do it than to do it wrong. How it worked (and will work again) is that source groups will dispatch a signal which will eventually result in a method on your cachefile strategy being called. That method can delete the file (or schedule it for deletion or do whatever it wants with it.

If you're looking for a way to do this by hand in the meantime, you can create ImageCacheFiles or LazyImageCacheFiles and calling their delete() method.

[davidtgq]

I'm looking for a solution to this as well, but I'm rather inexperienced so I'm not sure how to implement in code what I want to do in theory.

Basically, I'm looking for the opposite of generateimages. That is, it takes the specs and uses the same code that works out what the path should be, then try deleting the image at that path whether or not it exists. I think that's what you were suggesting with LazyImageCacheFiles, but I don't know how to use it, though it seems like it should be quite doable.

This is what I'm currently doing with a different app, I'm trying to find a way to achieve the same functionality with Imagekit:

@receiver(models.signals.post_save, sender=Image)
def warm_image(sender, instance, **kwargs):
    image_warmer = VersatileImageFieldWarmer(
        instance_or_queryset=instance,
        rendition_key_set='places',
        image_attr='image'
    )
    num_created, failed_to_create = image_warmer.warm()

@receiver(models.signals.post_delete, sender=Image)
def delete_image(sender, instance, **kwargs):
    instance.image.delete_all_created_images()
    instance.image.delete()

[tino]

@matthewwithanm what do you mean by:

If you're looking for a way to do this by hand in the meantime, you can create ImageCacheFiles or LazyImageCacheFiles and calling their delete() method.

ImageCacheFile (https://github.com/matthewwithanm/django-imagekit/blob/6457cf0c55dda6e59bd1af639e2c89b7f7af67d6/imagekit/cachefiles/__init__.py#L12) doesn't have a delete() method.

I am looking for a way to delete the cache file so it is regenerated

[vesterbaek]

@matthewwithanm Any progress on deletion? It being a manual process is fine -- IMO we just need some way of deleting all files generated of a source image field

[shepelevstas]

It there a change to have the delete() function?

[sam-dieck]

What has happened with this functionality? It's being several years.... what's the current recommended way to clean up the cache ?

[vstoykov]

Currently the only way to clear the cache is by deleting the directory configured in IMAGEKIT_CACHEFILE_DIR which will cause regeneration of all images on next access.

Also because in ImageKit 4.0 django cache backend is configured to preserve the state of images forever then you need to clear it also.

from imagekit.utils import get_cache

get_cache().clear()

If you are using version of ImageKit < 4.0 and you have configured some caching for ImageKit as described in the docs then you need to manually clear the configured cache because imagekit.utils.get_cahe function was not available prior to 4.0.

This procedure will clear the whole cache not only cached thumbnails for given image.

At the moment there is no functionality to clear specific thumbnail. If someone wants to contribute I will be very glad.

[shepelevstas]

I tryed to just delete cache directory, but then new cache image is not regenerated (after it has been generated for the first time), and I just get an error. What do I do to allow regeneration of missing cached images?

[vstoykov]

As I mentioned in the previous comment after you deleted cached images in the folder you need to clear the cache that stores if the file is generated or not. And this can be done using the commands above.

There is one caveat though. If you have not configured any cache (settings.CACHES), then Django use locmem by default which is a cache per thread. It's not very efficient for production and you should change it to something like memcached or redis.

If your cache is locmem then when you execute the commands in the shell for clearing the cache it will clear the cache only in the current thread (shell thread) and not the web workers threads. In order to clear theirs cache you need to restart the application.

For that reason for production is better to change the cache to some "external" service like memcached or redis.

For development you can just restart your server after clearing cache folder (because probably you do not have any cache configured in your development settings which means that locmem is used by default).

[jochengcd]

I am using the following to delete the cached files, e.g. cached_image is image.scaled_image. This deletes the cached image, if it does not exist storage.delete() does not generate an error, so no need to check if the file exists via .exists(). Afterwards then the cache-status is reset. Some target storage system may not support deletion, no idea what would happen then.

def _clear_image_cache(cached_image):
    cached_image.storage.delete(cached_image)
    cached_image.cachefile_backend.set_state(cached_image,
                                             CacheFileState.DOES_NOT_EXIST)

[jaap3]

@jochengcd Thanks, that looks super useful

It would be nice for BaseIKFile and ImageCacheFile to implement a .delete method so they can remove themself from storage \ cachefile_backend.

This way e.g. a models post_delete signal handler can take care of deleting the source and all derivatives of an image. For me this is an essential feature as otherwise it's hard to guarantee to a user that her/his images have been fully deleted.

[jaap3]

Based on @jochengcd I came up with this way to make sure the original and derived images are removed:

class MyImage(models.Model):
    file = models.ImageField()
    thumbnail = ImageSpecField(source='file', processors=[...])
    some_size = ImageSpecField(source='file', processors=[...])

    @staticmethod
    def post_delete(sender, instance, **kwargs):
        for field in ['thumbnail', 'some_size']:
            field = getattr(instance, field)
            try:
                file = field.file
           except FileNotFoundError:
                pass
           else:
                cache_backend = field.cachefile_backend
                cache_backend.cache.delete(cache_backend.get_key(file))
                field.storage.delete(file)
        instance.file.delete(save=False)

post_delete.connect(MyImage.post_delete, sender=MyImage)

Note that I don't use cachefile_backend.set_state but remove the key from the underlying cache instead. This way no information about the file remains at all.

I guess the FileNotFoundError handling might work for local (filesystem backed) backends only, I haven't tested this with s3 or anything.

All in all, it would be much nicer if this was abstracted away so a user can simply call instance.thumbnail.delete() and have ImageKit do the deleting, similar to a normal FileField.

[synycboom]

@jaap3 If you want it to work with s3, you have to change the line field.storage.delete(file) to field.storage.delete(file.name). Since s3 storage delete() method accepts only a string parameter.

[lifenautjoe]

Based on @jochengcd I came up with this way to make sure the original and derived images are removed:

class MyImage(models.Model):
    file = models.ImageField()
    thumbnail = ImageSpecField(source='file', processors=[...])
    some_size = ImageSpecField(source='file', processors=[...])

    @staticmethod
    def post_delete(sender, instance, **kwargs):
        for field in ['thumbnail', 'some_size']:
            field = getattr(instance, field)
            try:
                file = field.file
           except FileNotFoundError:
                pass
           else:
                cache_backend = field.cachefile_backend
                cache_backend.cache.delete(cache_backend.get_key(file))
                field.storage.delete(file)
        instance.file.delete(save=False)

post_delete.connect(MyImage.post_delete, sender=MyImage)

Note that I don't use cachefile_backend.set_state but remove the key from the underlying cache instead. This way no information about the file remains at all.

I guess the FileNotFoundError handling might work for local (filesystem backed) backends only, I haven't tested this with s3 or anything.

All in all, it would be much nicer if this was abstracted away so a user can simply call instance.thumbnail.delete() and have ImageKit do the deleting, similar to a normal FileField.

Tried it, says cachefile_backend does not exist 😢

[lifenautjoe]

For anyone wondering, I ended up passing the field to the following function

def delete_image_kit_image_field(image_kit_field):
    # ImageKit has a bug where files are cached and not deleted right away
    # https://github.com/matthewwithanm/django-imagekit/issues/229#issuecomment-315690575

    if not image_kit_field:
        return

    try:
        file = image_kit_field.file
    except FileNotFoundError:
        pass
    else:
        cache = get_cache()
        cache.delete(cache.get(file))
        image_kit_field.storage.delete(file.name)

    image_kit_field.delete()

[jaymzcd]

For anyone popping up having some issues with clearing and if you're using redis at least, I was able to clear just imagekit thumbnails, which then recreated them on demand as required by passing the prefix into the pattern:

from imagekit.utils import get_cache
get_cache().delete_pattern('imagekit:*')

This is supported in django-redis 4.1.0+

[99aman]

This code successfully delete images from CACHE folder(Django-3.0.1,windows 10)

class MyImage(models.Model):
     title      = models.Charfield(max_length=100)
     main_image = models.ImageField(upload_to='your_path')
     image_thmb = ImageSpecField(source="main_image",processor=...)

    def delete(self,*args,**kwargs):
            try:
                file1 = self.main_image.path
                file2 = self.image_thmb.path
                lst=[file1,file2] # put file1 and file2 in list
            except:
                pass
            else:
                for path in lst:
                    os.remove(path)
                self.blog_image.delete()
            super().delete(*args,**kwargs)```

[morlandi]

Best effort to regenerate the thumbnail when the associated image is updated

In a django Model, I keep an ImageField "image" and an associated "thumbnail", based on from imagekit.models.ImageSpecField.

I might need to rebuild the image, keeping the same filename for the bitmap.

Obviously, the thumbnail needs to be updated as well; my first attempt was:

self.thumbnail.generate(force=True)

but that's not enough; despite the force=True parameter, the thumbnail remains unchanged.

After some trials and errors, I ended up up with the following workaround, based on the previous comments on this issue:

class MyModel(model.Model):

    ...

    image = models.ImageField(_('Image'), null=True, blank=True, upload_to=get_acquisition_media_file_path)
    thumbnail = ImageSpecField(source='image', processors=[ResizeToFill(200, 100)], format='PNG')

    ...

    def save_image_from_data(self):

        # Create a new random image
        from random import randint
        rgb = (randint(0, 255), randint(0, 255), randint(0, 255))
        image_size = (200, 200)
        image = PILImage.new('RGB', image_size, rgb)
        with io.BytesIO() as output:
            image.save(output, format="PNG")
            contents = output.getvalue()

        # Best effort to remove the thumbnail, if any
        try:
            file = self.thumbnail.file
        #except FileNotFoundError:
        except:
            pass
        try:
            cache_backend = self.thumbnail.cachefile_backend
            cache_backend.cache.delete(cache_backend.get_key(file))
            #self.thumbnail.storage.delete(file)
            self.thumbnail.storage.delete(file.name)
        except:
            pass

        # Save image
        self.image = ContentFile(contents, "image.png")
        self.save()

        # Regenerate thumbnail
        self.thumbnail.generate(force=True)

        return True

I'm not fully happy with it, but it works