Closed fgd007 closed 11 months ago
On a further update: Some of these jobs fail. For instance when a image is very big (7000px x 4000px). I don't get it why this should be problem? Imgix should deal with that? A thumbnail version of the image in the Asset library loads an Imgix version successfully...
The failing asset shows: "This is not an image that can be manipulated"
ID 41896 Status Failed Error Resize operation failed Progress 0% Time to reserve 300 seconds Priority 1024 Pushed at Nov 20, 2023, 10:49:10 PM Updated at Nov 20, 2023, 10:52:24 PM Failed at Nov 20, 2023, 10:52:27 PM
Job Data
{
"description": "Optimizing image id 15608",
"criteria": {
"id": 15608,
"status": null
},
"fieldId": null,
"force": false
}
Can you please check your logs, or the details from the Queue utility, and give me the actual errors that are being logged?
Also did you update your version of nystudio107/craft-imageoptimize-imgix
to the latest version?
Running ImageOptimize 4.0.5. I bumped version of craft-imageoptimize-imgix to 4.0.3 manually. It didn't change anything.
When I run ddev craft image-optimize/optimize/create-asset
on certain images I get this:
2023-11-22 14:45:38 [queue.INFO] [craft\queue\QueueLogBehavior::beforeExec] [45622] Optimizing image id 254570 (attempt: 1, pid: 7710) - Started {"memory":5934592}
2023-11-22 14:45:38 [queue.INFO] [craft\queue\QueueLogBehavior::afterExec] [45622] Optimizing image id 254570 (attempt: 1, pid: 7710) - Done (time: 0.074s) {"memory":7518648}
When I run ddev craft image-optimize/optimize/create-asset
on other images, it is logging these steps:
2023-11-22 14:39:19 [queue.INFO] [craft\queue\QueueLogBehavior::beforeExec] [45621] Optimizing image id 16029 (attempt: 1, pid: 7613) - Started {"memory":5934592}
2023-11-22 14:39:19 [queue.INFO] [nystudio107\imageoptimize\services\OptimizedImages::shouldCreateVariants] Array
(
[s3] => *
)
{"memory":7737512}
2023-11-22 14:39:19 [queue.INFO] [nystudio107\imageoptimize\services\OptimizedImages::shouldCreateVariants] Matched '*' wildcard {"memory":7767872}
2023-11-22 14:39:19 [queue.INFO] [nystudio107\imageoptimize\services\OptimizedImages::addVariantImageToModel] URL created: https://XXX.imgix.net/uploads/DSC7432.jpg?auto=format&crop=focalpoint&domain=XXX.imgix.net&fit=crop&fp-x=0.5&fp-y=0.5&h=338&ixlib=php-3.3.1&q=82&w=480 {"memory":8025992}
2023-11-22 14:39:19 [queue.INFO] [nystudio107\imageoptimize\services\OptimizedImages::generatePlaceholders] generatePlaceholders for: nystudio107\imageoptimize\models\OptimizedImage Object
(
[_events:yii\base\Component:private] => Array
(
)
[_eventWildcards:yii\base\Component:private] => Array
(
)
[_behaviors:yii\base\Component:private] => Array
(
)
[_errors:yii\base\Model:private] =>
[_validators:yii\base\Model:private] =>
[_scenario:yii\base\Model:private] => default
[optimizedImageUrls] => Array
(
[480] => https://XXX.imgix.net/uploads/DSC7432.jpg?auto=format&crop=focalpoint&domain=XXX.imgix.net&fit=crop&fp-x=0.5&fp-y=0.5&h=338&ixlib=php-3.3.1&q=82&w=480
)
[optimizedWebPImageUrls] => Array
(
[480] => https://XXX.imgix.net/uploads/DSC7432.jpg?crop=focalpoint&domain=XXX.imgix.net&fit=crop&fm=webp&fp-x=0.5&fp-y=0.5&h=338&ixlib=php-3.3.1&q=82&w=480
)
[variantSourceWidths] => Array
(
[0] => 480
)
[variantHeights] => Array
(
[480] => 338
)
[focalPoint] => Array
(
[x] => 0.5
[y] => 0.5
)
[originalImageWidth] => 1024
[originalImageHeight] => 723
[placeholder] =>
[placeholderSvg] =>
[colorPalette] => Array
(
)
[lightness] => 0
[placeholderWidth] => 480
[placeholderHeight] => 338
[stickyErrors] => Array
(
)
)
{"memory":8031592}
2023-11-22 14:39:19 [queue.INFO] [nystudio107\imageoptimize\services\Placeholder::createTempPlaceholderImage] Creating temporary placeholder image for asset {"memory":8032768}
2023-11-22 14:39:19 [queue.INFO] [nystudio107\imageoptimize\services\Placeholder::generatePlaceholderImage] Generating placeholder image for asset {"memory":8260096}
2023-11-22 14:39:19 [queue.INFO] [nystudio107\imageoptimize\services\Placeholder::generateColorPalette] Generating color palette for: /var/www/html/storage/runtime/temp/16029655e1297295c84.12705360.jpg {"memory":8267728}
2023-11-22 14:39:19 [queue.INFO] [nystudio107\imageoptimize\services\OptimizedImages::addVariantImageToModel] Created transforms for variant: Array
(
[width] => 480
[useAspectRatio] =>
[aspectRatioX] => 4
[aspectRatioY] => 3
[retinaSizes] => Array
(
[0] => 1
)
[quality] => 82
[format] =>
)
{"memory":8358456}
2023-11-22 14:39:19 [queue.INFO] [nystudio107\imageoptimize\services\OptimizedImages::addVariantImageToModel] URL created: https://XXX.imgix.net/uploads/DSC7432.jpg?auto=format&crop=focalpoint&domain=XXX.imgix.net&fit=crop&fp-x=0.5&fp-y=0.5&h=564&ixlib=php-3.3.1&q=82&usm=20&w=800 {"memory":8360328}
2023-11-22 14:39:19 [queue.INFO] [nystudio107\imageoptimize\services\OptimizedImages::addVariantImageToModel] Created transforms for variant: Array
(
[width] => 800
[useAspectRatio] =>
[aspectRatioX] => 4
[aspectRatioY] => 3
[retinaSizes] => Array
(
[0] => 1
)
[quality] => 82
[format] =>
)
{"memory":8361816}
2023-11-22 14:39:19 [queue.INFO] [nystudio107\imageoptimize\services\OptimizedImages::addVariantImageToModel] URL created: https://XXX.imgix.net/uploads/DSC7432.jpg?auto=format&crop=focalpoint&domain=XXX.imgix.net&fit=crop&fp-x=0.5&fp-y=0.5&h=903&ixlib=php-3.3.1&q=82&usm=20&w=1280 {"memory":8363672}
2023-11-22 14:39:19 [queue.INFO] [nystudio107\imageoptimize\services\OptimizedImages::addVariantImageToModel] Created transforms for variant: Array
(
[width] => 1280
[useAspectRatio] =>
[aspectRatioX] => 16
[aspectRatioY] => 9
[retinaSizes] => Array
(
[0] => 1
)
[quality] => 82
[format] =>
)
{"memory":8365160}
2023-11-22 14:39:19 [queue.INFO] [nystudio107\imageoptimize\services\OptimizedImages::addVariantImageToModel] URL created: https://XXX.imgix.net/uploads/DSC7432.jpg?auto=format&crop=focalpoint&domain=XXX.imgix.net&fit=crop&fp-x=0.5&fp-y=0.5&h=1355&ixlib=php-3.3.1&q=82&usm=20&w=1920 {"memory":8367016}
2023-11-22 14:39:19 [queue.INFO] [nystudio107\imageoptimize\services\OptimizedImages::addVariantImageToModel] Created transforms for variant: Array
(
[width] => 1920
[useAspectRatio] =>
[aspectRatioX] => 16
[aspectRatioY] => 9
[retinaSizes] => Array
(
[0] => 1
)
[quality] => 82
[format] =>
)
{"memory":8368504}
2023-11-22 14:39:19 [queue.INFO] [nystudio107\imageoptimize\services\OptimizedImages::addVariantImageToModel] URL created: https://XXX.imgix.net/uploads/DSC7432.jpg?auto=format&crop=focalpoint&domain=XXX.imgix.net&fit=crop&fp-x=0.5&fp-y=0.5&h=500&ixlib=php-3.3.1&q=82&usm=20&w=500 {"memory":8370360}
2023-11-22 14:39:19 [queue.INFO] [nystudio107\imageoptimize\services\OptimizedImages::addVariantImageToModel] Created transforms for variant: Array
(
[width] => 500
[useAspectRatio] => 1
[aspectRatioX] => 1
[aspectRatioY] => 1
[retinaSizes] => Array
(
[0] => 1
[1] => 2
)
[quality] => 82
[format] =>
)
{"memory":8377480}
2023-11-22 14:39:19 [queue.INFO] [nystudio107\imageoptimize\services\OptimizedImages::addVariantImageToModel] URL created: https://XXX.imgix.net/uploads/DSC7432.jpg?auto=format&crop=focalpoint&domain=XXX.imgix.net&fit=crop&fp-x=0.5&fp-y=0.5&h=1000&ixlib=php-3.3.1&q=61&usm=20&w=1000 {"memory":8379336}
2023-11-22 14:39:19 [queue.INFO] [nystudio107\imageoptimize\services\OptimizedImages::addVariantImageToModel] Created transforms for variant: Array
(
[width] => 500
[useAspectRatio] => 1
[aspectRatioX] => 1
[aspectRatioY] => 1
[retinaSizes] => Array
(
[0] => 1
[1] => 2
)
[quality] => 82
[format] =>
)
{"memory":8380824}
2023-11-22 14:39:19 [queue.INFO] [craft\queue\QueueLogBehavior::afterExec] [45621] Optimizing image id 16029 (attempt: 1, pid: 7613) - Done (time: 0.250s) {"memory":8496520}
The placeholder images are generated via PHP regardless of your transform method, and it looks like your PHP install is running out of memory attempting to create the placeholder images for this very large image.
If you're not using the placeholder images, you can disable their generation via the config file:
https://github.com/nystudio107/craft-imageoptimize/blob/develop-v4/src/config.php#L58
When I disable all settings, it still tries to transform images locally. I don't understand why Imagemagick / local transforms are needed when all transforms can be done by Imgix. Can you clarify @khalwat?
2023-11-22 18:57:19 [queue.ERROR] [Imagine\Exception\RuntimeException] ImagickException: cache resources exhausted `/var/www/html/storage/runtime/temp/Wavy_Tech-01_Single-08.delimiter.655e4f0ccf8146.42562151.jpg' @ error/cache.c/OpenPixelCache/4095 in /var/www/html/vendor/pixelandtonic/imagine/src/Imagick/Imagick.php:168
Stack trace:
#0 /var/www/html/vendor/pixelandtonic/imagine/src/Imagick/Imagick.php(168): Imagick->sampleImage(10000.0, 10000.0)
#1 /var/www/html/vendor/pixelandtonic/imagine/src/Imagick/Imagick.php(34): Imagine\Imagick\Imagick->_thumbnailImage(2000.0, 2000.0, false, false, 3, true, true)
#2 /var/www/html/vendor/pixelandtonic/imagine/src/Imagick/Image.php(320): Imagine\Imagick\Imagick->smartResize(2000, 2000, true, true, 100)
#3 /var/www/html/vendor/craftcms/cms/src/image/Raster.php(459): Imagine\Imagick\Image->smartResize(Object(Imagine\Image\Box), true, true, 100)
#4 /var/www/html/vendor/craftcms/cms/src/image/Raster.php(261): craft\image\Raster->resize(2000, 2000)
#5 /var/www/html/vendor/craftcms/cms/src/helpers/ImageTransforms.php(378): craft\image\Raster->scaleToFit(2000, 2000, false)
#6 /var/www/html/vendor/craftcms/cms/src/helpers/ImageTransforms.php(202): craft\helpers\ImageTransforms::storeLocalSource('/var/www/html/s...', '/var/www/html/s...')
#7 /var/www/html/vendor/nystudio107/craft-imageoptimize/src/services/Placeholder.php(246): craft\helpers\ImageTransforms::getLocalImageSource(Object(craft\elements\Asset))
#8 /var/www/html/vendor/nystudio107/craft-imageoptimize/src/services/Placeholder.php(227): nystudio107\imageoptimize\services\Placeholder->createImageFromAsset(Object(craft\elements\Asset), 300, 300, 75, Array)
#9 /var/www/html/vendor/nystudio107/craft-imageoptimize/src/services/OptimizedImages.php(464): nystudio107\imageoptimize\services\Placeholder->createTempPlaceholderImage(Object(craft\elements\Asset), 1.0, Array)
#10 /var/www/html/vendor/nystudio107/craft-imageoptimize/src/services/OptimizedImages.php(576): nystudio107\imageoptimize\services\OptimizedImages->generatePlaceholders(Object(craft\elements\Asset), Object(nystudio107\imageoptimize\models\OptimizedImage), 1.0)
#11 /var/www/html/vendor/nystudio107/craft-imageoptimize/src/services/OptimizedImages.php(146): nystudio107\imageoptimize\services\OptimizedImages->addVariantImageToModel(Object(craft\elements\Asset), Object(nystudio107\imageoptimize\models\OptimizedImage), Object(craft\models\ImageTransform), Array, 1.0)
#12 /var/www/html/vendor/nystudio107/craft-imageoptimize/src/services/OptimizedImages.php(281): nystudio107\imageoptimize\services\OptimizedImages->populateOptimizedImageModel(Object(craft\elements\Asset), Array, Object(nystudio107\imageoptimize\models\OptimizedImage), false)
#13 /var/www/html/vendor/nystudio107/craft-imageoptimize/src/jobs/ResaveOptimizedImages.php(113): nystudio107\imageoptimize\services\OptimizedImages->updateOptimizedImageFieldData(Object(nystudio107\imageoptimize\fields\OptimizedImages), Object(craft\elements\Asset), false)
#14 /var/www/html/vendor/yiisoft/yii2-queue/src/Queue.php(243): nystudio107\imageoptimize\jobs\ResaveOptimizedImages->execute(Object(craft\queue\Queue))
#15 /var/www/html/vendor/yiisoft/yii2-queue/src/cli/Queue.php(147): yii\queue\Queue->handleMessage(45735, 'O:52:"nystudio1...', 300, 1)
#16 /var/www/html/vendor/craftcms/cms/src/queue/Queue.php(190): yii\queue\cli\Queue->handleMessage(45735, 'O:52:"nystudio1...', 300, 1)
#17 /var/www/html/vendor/craftcms/cms/src/queue/Queue.php(165): craft\queue\Queue->executeJob()
#18 [internal function]: craft\queue\Queue->craft\queue\{closure}(Object(Closure))
#19 /var/www/html/vendor/yiisoft/yii2-queue/src/cli/Queue.php(114): call_user_func(Object(Closure), Object(Closure))
#20 /var/www/html/vendor/craftcms/cms/src/queue/Queue.php(163): yii\queue\cli\Queue->runWorker(Object(Closure))
#21 /var/www/html/vendor/nystudio107/craft-imageoptimize/src/console/controllers/OptimizeController.php(122): craft\queue\Queue->run()
#22 /var/www/html/vendor/nystudio107/craft-imageoptimize/src/console/controllers/OptimizeController.php(98): nystudio107\imageoptimize\console\controllers\OptimizeController->runCraftQueue()
#23 [internal function]: nystudio107\imageoptimize\console\controllers\OptimizeController->actionCreateAsset(2706)
#24 /var/www/html/vendor/yiisoft/yii2/base/InlineAction.php(57): call_user_func_array(Array, Array)
#25 /var/www/html/vendor/yiisoft/yii2/base/Controller.php(178): yii\base\InlineAction->runWithParams(Array)
#26 /var/www/html/vendor/yiisoft/yii2/console/Controller.php(180): yii\base\Controller->runAction('create-asset', Array)
#27 /var/www/html/vendor/yiisoft/yii2/base/Module.php(552): yii\console\Controller->runAction('create-asset', Array)
#28 /var/www/html/vendor/yiisoft/yii2/console/Application.php(180): yii\base\Module->runAction('image-optimize/...', Array)
#29 /var/www/html/vendor/craftcms/cms/src/console/Application.php(91): yii\console\Application->runAction('image-optimize/...', Array)
#30 /var/www/html/vendor/yiisoft/yii2/console/Application.php(147): craft\console\Application->runAction('image-optimize/...', Array)
#31 /var/www/html/vendor/craftcms/cms/src/console/Application.php(122): yii\console\Application->handleRequest(Object(craft\console\Request))
#32 /var/www/html/vendor/yiisoft/yii2/base/Application.php(384): craft\console\Application->handleRequest(Object(craft\console\Request))
#33 /var/www/html/craft(15): yii\base\Application->run()
#34 {main}
Next Imagine\Exception\RuntimeException: Resize operation failed in /var/www/html/vendor/pixelandtonic/imagine/src/Imagick/Image.php:325
Stack trace:
#0 /var/www/html/vendor/craftcms/cms/src/image/Raster.php(459): Imagine\Imagick\Image->smartResize(Object(Imagine\Image\Box), true, true, 100)
#1 /var/www/html/vendor/craftcms/cms/src/image/Raster.php(261): craft\image\Raster->resize(2000, 2000)
#2 /var/www/html/vendor/craftcms/cms/src/helpers/ImageTransforms.php(378): craft\image\Raster->scaleToFit(2000, 2000, false)
#3 /var/www/html/vendor/craftcms/cms/src/helpers/ImageTransforms.php(202): craft\helpers\ImageTransforms::storeLocalSource('/var/www/html/s...', '/var/www/html/s...')
#4 /var/www/html/vendor/nystudio107/craft-imageoptimize/src/services/Placeholder.php(246): craft\helpers\ImageTransforms::getLocalImageSource(Object(craft\elements\Asset))
#5 /var/www/html/vendor/nystudio107/craft-imageoptimize/src/services/Placeholder.php(227): nystudio107\imageoptimize\services\Placeholder->createImageFromAsset(Object(craft\elements\Asset), 300, 300, 75, Array)
#6 /var/www/html/vendor/nystudio107/craft-imageoptimize/src/services/OptimizedImages.php(464): nystudio107\imageoptimize\services\Placeholder->createTempPlaceholderImage(Object(craft\elements\Asset), 1.0, Array)
#7 /var/www/html/vendor/nystudio107/craft-imageoptimize/src/services/OptimizedImages.php(576): nystudio107\imageoptimize\services\OptimizedImages->generatePlaceholders(Object(craft\elements\Asset), Object(nystudio107\imageoptimize\models\OptimizedImage), 1.0)
#8 /var/www/html/vendor/nystudio107/craft-imageoptimize/src/services/OptimizedImages.php(146): nystudio107\imageoptimize\services\OptimizedImages->addVariantImageToModel(Object(craft\elements\Asset), Object(nystudio107\imageoptimize\models\OptimizedImage), Object(craft\models\ImageTransform), Array, 1.0)
#9 /var/www/html/vendor/nystudio107/craft-imageoptimize/src/services/OptimizedImages.php(281): nystudio107\imageoptimize\services\OptimizedImages->populateOptimizedImageModel(Object(craft\elements\Asset), Array, Object(nystudio107\imageoptimize\models\OptimizedImage), false)
#10 /var/www/html/vendor/nystudio107/craft-imageoptimize/src/jobs/ResaveOptimizedImages.php(113): nystudio107\imageoptimize\services\OptimizedImages->updateOptimizedImageFieldData(Object(nystudio107\imageoptimize\fields\OptimizedImages), Object(craft\elements\Asset), false)
#11 /var/www/html/vendor/yiisoft/yii2-queue/src/Queue.php(243): nystudio107\imageoptimize\jobs\ResaveOptimizedImages->execute(Object(craft\queue\Queue))
#12 /var/www/html/vendor/yiisoft/yii2-queue/src/cli/Queue.php(147): yii\queue\Queue->handleMessage(45735, 'O:52:"nystudio1...', 300, 1)
#13 /var/www/html/vendor/craftcms/cms/src/queue/Queue.php(190): yii\queue\cli\Queue->handleMessage(45735, 'O:52:"nystudio1...', 300, 1)
#14 /var/www/html/vendor/craftcms/cms/src/queue/Queue.php(165): craft\queue\Queue->executeJob()
#15 [internal function]: craft\queue\Queue->craft\queue\{closure}(Object(Closure))
#16 /var/www/html/vendor/yiisoft/yii2-queue/src/cli/Queue.php(114): call_user_func(Object(Closure), Object(Closure))
#17 /var/www/html/vendor/craftcms/cms/src/queue/Queue.php(163): yii\queue\cli\Queue->runWorker(Object(Closure))
#18 /var/www/html/vendor/nystudio107/craft-imageoptimize/src/console/controllers/OptimizeController.php(122): craft\queue\Queue->run()
#19 /var/www/html/vendor/nystudio107/craft-imageoptimize/src/console/controllers/OptimizeController.php(98): nystudio107\imageoptimize\console\controllers\OptimizeController->runCraftQueue()
#20 [internal function]: nystudio107\imageoptimize\console\controllers\OptimizeController->actionCreateAsset(2706)
#21 /var/www/html/vendor/yiisoft/yii2/base/InlineAction.php(57): call_user_func_array(Array, Array)
#22 /var/www/html/vendor/yiisoft/yii2/base/Controller.php(178): yii\base\InlineAction->runWithParams(Array)
#23 /var/www/html/vendor/yiisoft/yii2/console/Controller.php(180): yii\base\Controller->runAction('create-asset', Array)
#24 /var/www/html/vendor/yiisoft/yii2/base/Module.php(552): yii\console\Controller->runAction('create-asset', Array)
#25 /var/www/html/vendor/yiisoft/yii2/console/Application.php(180): yii\base\Module->runAction('image-optimize/...', Array)
#26 /var/www/html/vendor/craftcms/cms/src/console/Application.php(91): yii\console\Application->runAction('image-optimize/...', Array)
#27 /var/www/html/vendor/yiisoft/yii2/console/Application.php(147): craft\console\Application->runAction('image-optimize/...', Array)
#28 /var/www/html/vendor/craftcms/cms/src/console/Application.php(122): yii\console\Application->handleRequest(Object(craft\console\Request))
#29 /var/www/html/vendor/yiisoft/yii2/base/Application.php(384): craft\console\Application->handleRequest(Object(craft\console\Request))
#30 /var/www/html/craft(15): yii\base\Application->run()
#31 {main} {"memory":9700592,"exception":"[object] (Imagine\\Exception\\RuntimeException(code: 445): Resize operation failed at /var/www/html/vendor/pixelandtonic/imagine/src/Imagick/Image.php:325)
[previous exception] [object] (ImagickException(code: 445): cache resources exhausted `/var/www/html/storage/runtime/temp/Wavy_Tech-01_Single-08.delimiter.655e4f0ccf8146.42562151.jpg' @ error/cache.c/OpenPixelCache/4095 at /var/www/html/vendor/pixelandtonic/imagine/src/Imagick/Imagick.php:168)"}
My bad -- it looks like that setting may only be exposed in the config file:
https://github.com/nystudio107/craft-imageoptimize/blob/develop-v4/src/config.php#L59
Ah, I was too quick and thought it was the setting Create Placeholder Silhouettes.
Verified this custom config setting, and it works now!
But to conclude with replying to my initial question: Yes, even when using Imgix as provider, it will spin up jobs in the queue.
Disabling the placeholders kills the color palette feature (see also issue #175).
Imgix is also able to generate a color palette for an asset, but I don't see this endpoint being used by craft-imageoptimize-imgix
. Would be a nice feature though.
Right because it's pre-generating the URLs so that it doesn't have to ping Imgix each time to generate them. Arguably that could be optimized.
Describe the bug
I have ImageOptimize installed, and moved the configuration to Imgix recently. However, when touching the images (via an import or whatever) I see loads of "Optimizing image id XXXX" jobs being fired.
They all fail. The whole image optimization step should be done by Imgix?Seems like a bug? I saw this issue which seemed like a similar issue, but that one should be fixed and merged already.
This is the content of one of those failed jobs:Edit: most jobs don't fail. They run without issues. But are they necessary?
To reproduce
Expected behaviour
I would expect that the image optimizing step is done by Imgix and that no additional job is necessary.
Screenshots
Versions