search

MilesCranmer / PySR

High-Performance Symbolic Regression in Python and Julia
https://astroautomata.com/PySR
Apache License 2.0
2.19k stars 207 forks source link

[BUG] Impossible to stop early on Jupyter #260

Open MilesCranmer opened 1 year ago

MilesCranmer commented 1 year ago

Right now it seems impossible to stop early within Jupyter notebooks. If one enters a search that takes a long time, the only way to quit is to restart the Jupyter kernel entirely. How might we be able to get early stopping working in Jupyter, through PyCall.jl? Ideally, hitting the stop button should be enough. However, hitting <ctl-c> in any PyJulia call does not work.

Would any of you happen to have ideas for implementing this; @mkitti @ngam @marius311?

A MWE is:

from julia import Main as jl
jl.eval("sleep(20)")

this cannot be stopped early inside IPython/Jupyter. It only can be stopped early within Python itself.

Maybe something like releasing the GIL after calling Julia... But that seems a bit dangerous.

marius311 commented 1 year ago

I forget if this is Julia specific or Jupyter, but there's a different behavior if you click stop once vs. 5 times in succession within a couple of seconds. Does the latter work?

mkitti commented 1 year ago

We would need to make sure the interrupt signal gets propagated. Does this work properly if we are running a Julia kernel directly?

MilesCranmer commented 1 year ago

Clicking stop quickly in succession does not seem enough to break out of the Julia call, unfortunately. @mkitti do you mean would stop the usual symbolic regression search directly from Julia REPL? In which case the answer is yes.

I wonder whether this behavior is seen in libraries which make lengthy and synchronous python calls to C++ backends? And if so, how they deal with it… Maybe it’s different because it is CPython and all.

mkitti commented 1 year ago

I mean does interrupt work correctly from a Jupyter IJulia kernel? https://github.com/JuliaLang/IJulia.jl

ngam commented 1 year ago

We would need to make sure the interrupt signal gets propagated

I guess by this, @mkitti means the interrupt signal must pass through the interface without vanishing. It seems that it vanishes somehow. I am not familiar with the codebase of PyCall :(

I mean does interrupt work correctly from a Jupyter IJulia kernel? https://github.com/JuliaLang/IJulia.jl

That will be besides the point because the MWE above is in the Python REPL...