progval / Limnoria

A robust, full-featured, and user/programmer-friendly Python IRC bot, with many existing plugins.
https://docs.limnoria.net/
Other
621 stars 174 forks source link

Stop using `fork` method to spawn processes #1600

Open progval opened 1 month ago

progval commented 1 month ago

Python 3.14 switches multiprocessing's default start method from fork to forkserver, which requires target functions to be picklable. This is because fork() is unsafe in threaded programs, like Limnoria.

However, Limnoria currently strongly depends on the fork() method (and explicitly opts into it in order to support Python >= 3.14), so that plugins can use commands.process() to call functions defined in plugins, despite plugins not being in Python's import path through:

https://github.com/progval/Limnoria/blob/54c09809786db7a6468c48dedc788287fbcded72/src/plugin.py

With the forkserver method, this wouldn't work because when a plugin calls commands.process(target=f) where f is a function defined in the plugin, the forkee would try to import the plugin (because it is needed to unpickle f), and fail.

Even if we could somehow make the forkee import all plugins and unpickle of Limnoria's state, this would be very wasteful, especially considering some plugins run commands.process() very often.