rails / solid_queue

Database-backed Active Job backend
MIT License
1.84k stars 110 forks source link

Fail in-progress jobs when the worker running them exits abnormally #277

Closed rosa closed 1 month ago

rosa commented 1 month ago

This applies to:

To do this easily, since the supervisor doesn't register all workers for efficiency, we need to rely on a new unique identifier that links the supervisor with their configured processes. Since the registration happens after forking, the supervisor doesn't know the registered process IDs of its supervised processes. This unique identifier is a name that gets randomly generated when the process is instantiated. This made me realise I was reusing the configured processes object to start new processes, which is quite prone to issues with already created thread pools and stuff like that 😬 Because of this, this PR also changes the approach to have the Configuration object return configured processes that need to be instantiated before starting, and each time create a new object.