search

datopian / ckanext-asset-storage

CKAN extension implementing IUploader interface for cloud storage of assets e.g. images. Intended for use with ckanext-blob-storage that provides direct to cloud storage of resource data.
MIT License
2 stars 2 forks source link

Replaced asset files are not removed from storage #4

Open shevron opened 4 years ago

shevron commented 4 years ago

Not sure if this is LocalStorage specific or not (I am guessing not) but when an asset it replaced, the old asset is not removed from storage. The expected behavior is that an uploaded asset is removed when it is replaced by a newer file, as this is the behavior of the default CKAN Uploader.

This probably happens because our assets start with http:// or https:// meaning they look to the uploader as if they are external assets and not uploaded, and should not be cleared.

We need smarter logic here, basically check if the file is uploaded by us and if so call the storage backend delete() method on the old file.

Note that the backend's delete functionality is tested to work properly. This is somewhere in the uploader logic.

shevron commented 4 years ago

This happens for Google Cloud Storage as well, although (at least for non-private assets) it may be for a different root cause.