search

thumbor-community / aws

Thumbor AWS extensions
MIT License
155 stars 70 forks source link

Feature request: modify S3 result storage path to make for easier deletion #148

Open karptonite opened 4 years ago

karptonite commented 4 years ago

For results storage in s3, the url for the requested image is translated directly into a storage path. This works fine, until you want to delete an image and all of its sizes, but don't know all the sizes you may have generated for that file. There is no fast way to search s3 for a filename if you don't know what folder it is in.

I have a suggestion for an option for making deletion easier. Say you request an image with the path: /UeaoGZaXaFY1PO3d-EQIBjpy5jI=/576x324/filters:strip_icc()/pic2212632.jpg Right now, it will go into s3 with the identical folder structure: /UeaoGZaXaFY1PO3d-EQIBjpy5jI=/576x324/filters:strip_icc()/pic2212632.jpg I propose prepending the filename to go an additional folder deep: pic2212632.jpg/UeaoGZaXaFY1PO3d-EQIBjpy5jI=/576x324/filters:strip_icc()/pic2212632.jpg Then, when it comes time to delete this image, you can just delete everything recursively in the folder pic2212632.jpg.

I may work on this in a branch myself, but with the coming move to Python 3, I'm not sure that it is worth trying to bring this back to the main repo, or if it is even a feature others would be interested in.

Any thoughts?

Bladrak commented 4 years ago

Hi, we've set this up previously to optimize fetching performances in S3 (it was in the official AWS architects recommandations back when we set up the project). Since then, this may have changed, but TBH I don't really have time to look it up.

kg-currenxie commented 3 years ago

Would love this as well! This could possibly be a small deal-breaker when having to delete account information due to GDPR etc..