[3.x]: 3.8.17

[jamie-s-white]

What happened?

Description

When re-indexing assets, if any image re-index fails (in this case because the image is too large and the job runs out of memory), then the asset is erroniously listed for deletion from the index.

Steps to reproduce

1. Upload an image for which the server does not have enough RAM to process
2. Navigate to /admin/utilities/asset-indexes
3. Ensure "Cache remote images" is checked
4. Invoke "Update asset indexes"

Expected behavior

Craft CMS should be clear that not all images were re-indexed, and not then list the images that weren't re-indexed as missing and ask if you want to remove them.

Actual behavior

Lists files as missing and asks if we want to remove them.

Screenshots of behaviour

Server erroring as it reaches images that it tries to process, but runs out of memory whilst doing so:

At the end of the process, for each image that the server failed to process, it then lists it for removal saying they couldn't be found, whilst in reality it did find the image, it is just the process crashed before it was fully processed:

Somewhat related: With Servd, the asset processing is offloaded to a different server, so the "Cache remote images" option doesn't actually do anything. We would like, for our Servd instances, to have this option removed or at least disabled by default, as now we have to remember to manually disable this option before we re-index our assets on Servd (otherwise we run into this bug).

Craft CMS version

3.8.17

PHP version

7.4.33

Operating system and version

servd.host instance

Database type and version

MariaDB 10.1.32

Image driver and version

imagick 3.7.0

Installed plugins and versions

"aelvan/imager": "2.4.0",
"aelvan/inlin": "2.1.1",
"charliedev/element-map": "1.2.1",
"clubstudioltd/craft-asset-rev": "6.0.2",
"craftcms/aws-s3": "1.3.2",
"craftcms/cms": "3.8.17",
"craftcms/feed-me": "4.7.0",
"craftcms/redactor": "2.10.12",
"doublesecretagency/craft-cpcss": "2.4.0",
"mmikkel/cp-field-inspect": "1.4.4",
"mmikkel/reasons": "2.3.1",
"nystudio107/craft-retour": "3.2.10",
"percipioglobal/craft-colour-swatches": "1.7.0",
"rias/craft-scout": "1.3.2",
"sebastianlenz/linkfield": "1.0.25",
"verbb/buttonbox": "3.1.1",
"supercool/connect": "2.0.0",
"supercool/dates-and-times": "2.0.1",
"supercool/event-manager": "2.4.6",
"supercool/fetch": "1.0.3",
"supercool/functions": "1.0.0",
"supercool/scheduler": "2.0.2.2",
"supercool/spektrix-fields": "1.0.2",
"topshelfcraft/wordsmith": "3.3.2",
"verbb/default-dashboard": "1.0.8",
"verbb/super-table": "2.7.5.1",
"vlucas/phpdotenv": "2.6.6",
"verbb/color-mixer": "2.0.1",
"servd/craft-asset-storage": "^2.8",
"verbb/tablemaker": "3.0.4",
"spicyweb/craft-odds-and-ends": "3.0.2"

[i-just]

Hi Jamie,

Thanks for getting in touch.

• I can only trigger a similar behaviour if I trigger an error after the asset element for the file was created but before the asset index is updated with its recordId
• I can only trigger a similar behaviour for Craft 3 and not Craft 4;

Asset indexing was improved in Craft 4. Under the same scenario, the asset would be reported as skipped, not listed as missing.

As per our chat, my suggestion would be to update the project to Craft 4 to get this and other asset-indexing benefits.

I’m going to go ahead and close it, but please feel free to get in touch if the issue still occurs on Craft 4.