SSG prematurely failing due to changes in spatie/fork package

globalexport commented 7 months ago

Is this really guaranteed to wait for all closures to finish?

 $results = $this->tasks->run(...$closures);

[EDIT] Please see: https://github.com/statamic/ssg/issues/155#issuecomment-1835763494

I am in the process of upgrading Statamic v3 to v4. For that reason, I am not 100% sure if the adjusted SSG setup I created in the past is responsible for a possible race condition. That's why I am testing both, the original SSG command and my custom one.

The original command gives me:

For the sake of simplicity, I am using the new shouldRejectPage() method to only allow a small portion of 10 blog pages` urls to be generated on 1 or 2 workers.

Using 1 worker always succeeds. Using 2 or more workers rarely succeeds (by chance).

In my own code I am logging everything (method entering, exceptions, etc..), but there is nothing indicating errors during the html generation.

What caught my eye in particular was that the process is interrupted directly after one worker returned from its closure method. There is no NotGeneratedException of any worker. It looks like other workers do not get the time to complete their task and return their arrays.

return compact('count', 'warnings', 'errors');

Therefore, worker data is missing and the process ends up in anyTasksFailed without any useful information.

if ($this->anyTasksFailed($results)) {      
  throw GenerationFailedException::withConsoleMessage("\x1B[1A\x1B[2K");
}

I am thankful for any ideas.

globalexport commented 7 months ago

> php please support:details

Environment
Application Name: Statamic
Laravel Version: 10.34.2
PHP Version: 8.3.0
Composer Version: 2.3.10
Environment: local
Debug Mode: ENABLED
URL: my-static.website.com/
Maintenance Mode: OFF

Cache
Config: NOT CACHED
Events: NOT CACHED
Routes: NOT CACHED
Views: CACHED

Drivers
Broadcasting: log
Cache: statamic
Database: mysql
Logs: stack / single
Mail: smtp
Queue: sync
Session: file

Statamic
Addons: 3
Antlers: runtime
Stache Watcher: Enabled
Static Caching: Disabled
Version: 4.31.0 PRO

Statamic Addons
spatie/statamic-responsive-images: 4.1.1
statamic/ssg: 2.2.0
my/cpssg: dev-feature/next

globalexport commented 7 months ago

I found out that the spatie/fork package is the reason for this behavior. After downgrading to v1.1.3 the problem is gone. They introduced a new way to kill tasks in their latest two versions (posix_kill) which seems to have an impact on the generation capabilities of Statamic's package.

https://github.com/spatie/fork/compare/1.1.3...1.2.0

globalexport commented 7 months ago

Maybe it is better to leave this issue open until there is a mitigation for further development?

statamic / ssg

SSG prematurely failing due to changes in spatie/fork package #155