Implement an image creation automatism

[websolutions-hamburg]

On normal web hosting packages you have a lot of php restrictions. For example the memory_limit or max_execution_time. If you have a image gallery with responsive size configuration e.q. with 1x,2x and 20 images you need a high memory_limit and high max_execution_time to genreate all the images.

What I observed, is that my test site is not available on first page call. Also the second page call will not show the page. After the third page call the site is available. I think, my web server, needs 3 * max_execution_time to generate all images on the test site.

It is not a problem to call the page 3 time on first creating. But what if I purge the image cache? Now I must check all pages manualy if all images are created.

I'm not 100% sure, but I think the image cache will not purge automaticly in Contao 4.4. So the problem is not so huge.

My question: Is it possible and useful to implement an image creation automatism. I think of something like the "Rebuild the search index". So if I purge the image cache, I can click after that on a button and Contao will open all visible pages and create the images. Contao open the page e.g. via ajax and create a defined number of images for each call. You can configure the maximum of images that will be create on one page call in the configurations.

I'm not sure if something like this is possible and if it make sense. So, what the core developers think about it?

(I hope it is ok to ask this question here in the issue tracker, because I'm not sure if the core developers will read the community forum.)

[leofeyer]

I think @aschempp has tried this already.

[aschempp]

I implemented an image cache warmer, but only for my custom module. The core does not and can not know which image is used where. At least not for all cases, we could only e.g. build a cache for image content elements.

I had multiple issues with it, mostly memory issues because apparently Imagine or Imagick (which is used on my server) has memory leaks.

[dmolineus]

As a workaround in Contao 3 you could rebuild the search index. Contao then calls all pages and the images are rendered. I didn't look up if Contao 4 works the same.

[websolutions-hamburg]

As a workaround in Contao 3 you could rebuild the search index. Contao then calls all pages and the images are rendered. I didn't look up if Contao 4 works the same.

Yes, this is a nice idea. But the problem is, on first run you will get a 500 error for a page with a high number of images. And so you must run the rebuild process again and you can get the same error for the same site again and again. On my current page, I need to run the rebuild process 3 times to get the success message.

[ausi]

I think we should move the resizing of the images from the main request to the image requests. This could work like so:

1. The image factory creates the resize configuration and the image path assets/images/X/foo-XXXXX.jpg and stores it in a cache or database (but doesn’t actually resize the image).
2. The browser sees the <img src="assets/images/X/foo-XXXXX.jpg"> and sends an HTTP request for the image.
3. The image doesn’t exist yet so a Symfony route for assets/images/* kicks in and reads from the cache or database if a resize configuration exists for this path.
4. The image gets resized, stored in assets/images and sent to the browser.
5. Every subsequent request to the image gets served directly from the webserver (without PHP).

This should make the main requests significantly faster and would prevent such 500 errors.

A problem with this approach could be that the file doesn’t exist after the image factory got called, so things like getimagesize() on the image path doesn’t work.

[bennyborn]

Any ETA on when this could be implemented? I'm currently working on a site which has galleries with lots of images (200-400 per gallery). At the moment this totals at around 10.000 images.

Generating the thumbnails in frontend is troublesome but thats okay.

The real problem here is the backend... When creating a new gallery element Contao tries to display thumbnails of them all to make them sortable, thats absolutely deadly. Same thing goes for the filemanager. If you accidentally opened multiple galleries you'll have a bad time.

An even bigger problem is the database synchronisation. I think it breaks just because of the huge amount of files :\

[ausi]

Note to myself:

• Use a cache (maybe stored in assets/images/pending)
• Use the locking (also somewhere in assets/images) component with a TTL (like 60s)
• File class has to check the cache for imagesize.

[ausi]

Any ETA on when this could be implemented?

On 21.06.2019 at 11:28 see https://github.com/contao/contao/commit/9dc7bad3ce7b55c284a9f32609e3e2def6b41e84 😉

[bennyborn]

On 21.06.2019 at 11:28 see contao/contao@9dc7bad 😉

Did you notice I asked this in 2018? I'm well aware that it has already been implemented :P