search

liip / LiipImagineBundle

Symfony Bundle to assist in image manipulation using the imagine library
http://liip.ch
MIT License
1.66k stars 378 forks source link

Add the option to use Symfony locks #1604

Open BentiGorlich opened 3 months ago

BentiGorlich commented 3 months ago

Is your feature request related to a problem? Please describe. When a user uploads a pretty large image that takes some time to generate a thumbnail for and said user keeps on hitting f5 on the page because the image is not loading (it is being generated) multiple processes try to generate the thumbnail, one for each reload. Because the image processing on its own can eat up RAM quite well you can run into the issue that you php-fpm process gets killed to recover memory, because otherwise the server would crash

Describe the solution you'd like Add the option to use the Symfony lock component so each image can only be processed by exactly one process at a time

Describe alternatives you've considered I though about triggering the generation on uploading the image and that might help, but it doesn't really solve the issue

Additional context https://github.com/MbinOrg/mbin

TheVillageGuy commented 3 months ago

Adding to this; because the new image is federated other users come in to look at the image as well within a matter of seconds (some servers don't make their own ocal copies but link directly to the new image) so there can easily be 10 users requesting and even F5ing the same image, pushing the server to its memory limit

dbu commented 2 months ago

there is integration with symfony messenger to pre-render images when they are uploaded. but of course that is asynchronous and therefore won't prevent your user from refreshing. we should be using (not sure how well we currently do) the symfony event system so that you could setup messenger to warmup and notify other people only when the cache warmup is done instead of immediately, if your project is all about images.

but nonetheless, using the lock component to prevent parallel warmups seems like a good idea. we are currently :tm: in the process of thinking up version 3 of this bundle, and aim to radically refactor everything to have clean and clear interfaces and workflows.

i added this issue to the 3.0 refactoring milestone so we keep this in mind too. if you want to get involved in the refactoring, the starting point would be #1590

BentiGorlich commented 2 months ago

Oh that sounds great. Maybe if the warmup messages would be sent to the sync transport it could actually be done on uploading...