vendure-ecommerce / vendure

The commerce platform with customization in its DNA.
https://www.vendure.io
Other
5.61k stars 993 forks source link

Purge derivated cached assets when the source is deleted #969

Open gkielwasser opened 3 years ago

gkielwasser commented 3 years ago

Is your feature request related to a problem? Please describe. Whenever we delete an asset, the derivated assets are not deleted from the cache folder.

Describe the solution you'd like At the same time we delete source asset, we should also delete derivated cached assets.

Additional context In case the assets are stored in the cloud it will avoid paying for deleted assets. It will also prevent to access an asset that has been deleted using the cached one.

michaelbromley commented 3 years ago

This is a bit of a tricky problem to solve in a generic fashion because each of the cached images has a unique filename like my-image__previewba31e112ce0b2079086a6b7d6150145f.jpg where the hash is unknowable to the AssetServerPlugin.

So to delete these images we would do something like this:

  1. Set up an EventBus subscription on the AssetEvent listening for the 'deleted' event type.
  2. Get the asset.preview identifier
  3. Delete all images from the cache location which have that preview identifier as a prefix.

Step 3 is where the current AssetStorageStrategy interface in insufficient. It has a deleteFile(identifier: string): Promise<void>; method, but that needs the exact filename, which we do not know due to the hash suffix.

The above process can still be set up as part of your custom code, as long as your file storage API provides a way for you to delete files based on a prefix (i.e. wildcard / regex deletion). For Amazon S3, something like include filters might work.

For the LocalStorageStrategy which uses Node's fs module, you could read the cache dir of the preview and inspect each filename and delete those that match the preview prefix.

I'm not yet sure about the best way to handle this in a general sense, but perhaps we could allow the AssetStorageStrategy.deleteFile() method to also accept a glob/regex. The challenge here is that then we need to be confident that the various persistence layers used by people (local fs, S3, Azure, etc) can be made to support this.