`craft blitz/cache/generate` completes without caching all pages

[ccchapman]

Support Request

We are experiencing an issue where craft blitz/cache/generate exits after caching 494 of 3411 total possible pages. We are using the LocalGenerator because the environment is not public.

$ time ./craft blitz/cache/generate
Generating Blitz cache...
[========>                                                   ] 14% (495/3411) ETA: 1944 sec.
Generated 494 of 3411 total possible pages and includes. To see why some pages were not cached, enable the `debug` config setting or use the `debug` flag and then open the Blitz log (in `storage/logs/blitz-****.log`, for example).
Blitz cache generation complete.

real    9m51.452s
user    0m2.471s
sys 0m10.345s

Every time we run the command, it stops when it hits this page number.

The issue does not happen when cache is generated through the queue: in this case, the number of cached pages is in the thousands. We suspect that the batching performed by the queued job could be masking the issue.

tail blitz-****.log shows the below when the issue happens:

[2024-11-19 17:08:15] Class "Amp\\Parallel\\Context\\ContextException" not found
[2024-11-19 17:08:15] Blitz cache generation complete. [via console command by "a5c5b759"]

We enabled debug and have messages for why a small handful of pages do not get cached (e.g. transform generation URLs) but believe they are insignificant and not the root cause. We have investigated the specific pages it would be attempting to cache around the time it exits and confirmed they are cacheable, functional pages.

We have looked into if there might be a reached limit on the number of file descriptors or limit on the number of child processes. The file descriptor limit is 1024 per process and it appears like the process is exiting before it reaches this limit. However, we have not fully ruled this out. After considering this being a possible culprit, we did find references on GitHub which reference it being a possible issue.¹

1: https://github.com/amphp/parallel/issues/112#issuecomment-620166329

Plugin Version

4.23.7

[bencroker]

The cache generation process is actually completing, as indicated by the message Blitz cache generation complete.. I’ll look into the progress bar issue, as it should really show 100% when it fully completes.

As for why only 494 of 3411 total possible pages are being cached, with the debug config setting enabled, you should see the exact reason for why pages were not cached. Can you ensure it is enabled, rerun the command and let me know what you see?

[ccchapman]

As for why only 494 of 3411 total possible pages are being cached, with the debug config setting enabled, you should see the exact reason for why pages were not cached. Can you ensure it is enabled, rerun the command and let me know what you see?

As I mentioned, it appears to be unrelated, we see a couple dozen messages (i.e. not hundreds or thousands) of this type:

[2024-11-20 12:37:18] Page not cached because it contains transform generation URLs. Consider setting the `generateTransformsBeforePageLoad` general config setting to `true` to fix this. [redacted]

[2024-11-20 12:41:34] Uncaught Twig\Error\RuntimeError in child process or thread with message "An exception has been thrown during the rendering of a template ("")." and code "0"; use Amp\Parallel\Sync\ContextPanicError::getOriginalTrace() for the stack trace in the child process or thread. Uncaught yii\web\ForbiddenHttpException in child process or thread with message "" and code "0"; use Amp\Parallel\Sync\ContextPanicError::getOriginalTrace() for the stack trace in the child process or thread [redacted]

I should note that the process continues to run for several minutes after we see the last of this type of message. The only error in the vicinity of the exit is the Amp one in the original comment.

For what it's worth, we do have generateTransformsBeforePageLoad enabled. The message seemed to be because the asset was missing on the filesystem.

[bencroker]

There could be a template issue that is causing an exception to be thrown, but that should be logged. I’ll look into why Amp\\Parallel\\Context\\ContextException is not found, and let you know what I find.