laravel / horizon

Dashboard and code-driven configuration for Laravel queues.
https://laravel.com/docs/horizon
MIT License
3.88k stars 659 forks source link

Ensure out of memory events are raised before exiting the process #1341

Closed jameshulse closed 1 year ago

jameshulse commented 1 year ago

Problem

I want to introduce extra logging into our application to log when a supervisor (or master) runs out of memory. I was struggling to get these events to fire despite lowering the memory limits to very small amounts.

From my testing it turns out that the $master->terminate(12); call will eventually call exitProcess on the Supervisor which calls to the system exit function. This means that the PHP process ends immediately and the line which raises the event is never hit.

image

Terminate calls exit

image

exit in turn calls exitProcess

Solution

The solution seems to be simply raising the event before terminating the process.

Other issues

It is also worth raising that the $supervisor instance isn't serialisable as it holds a reference to a Process instance. This meant that my first attempt to handle these events on a queue failed.