search

JuliaPy / PythonCall.jl

Python and Julia in harmony.
https://juliapy.github.io/PythonCall.jl/stable/
MIT License
810 stars 64 forks source link

Loading both `juliacall` and `pyjulia` triggers crash #440

Open MilesCranmer opened 9 months ago

MilesCranmer commented 9 months ago

I would not expect this to work anyways but just wanted to point it out:

[ins] In [1]: from julia import Main as pyjulia_jl

[ins] In [2]: from juliacall import Main as juliacall_jl
Fatal Python error: PyThreadState_Get: the function must be called with the GIL held, but the GIL is released (the current Python thread state is NULL)
Python runtime state: initialized

Thread 0x000000016e883000 (most recent call first):
  File "/Users/mcranmer/.pyenv/versions/3.10.10/lib/python3.10/threading.py", line 320 in wait
  File "/Users/mcranmer/.pyenv/versions/3.10.10/lib/python3.10/threading.py", line 607 in wait
  File "/Users/mcranmer/.pyenv/versions/3.10.10/lib/python3.10/site-packages/IPython/core/history.py", line 884 in run
  File "/Users/mcranmer/.pyenv/versions/3.10.10/lib/python3.10/site-packages/IPython/core/history.py", line 60 in only_when_enabled
  File "/Users/mcranmer/.pyenv/versions/3.10.10/lib/python3.10/site-packages/decorator.py", line 232 in fun
  File "/Users/mcranmer/.pyenv/versions/3.10.10/lib/python3.10/threading.py", line 1016 in _bootstrap_inner
  File "/Users/mcranmer/.pyenv/versions/3.10.10/lib/python3.10/threading.py", line 973 in _bootstrap

Current thread 0x00000001e0935ec0 (most recent call first):
  File "/Users/mcranmer/PermaDocuments/PySR/.venv/lib/python3.10/site-packages/juliacall/__init__.py", line 222 in init
  File "/Users/mcranmer/PermaDocuments/PySR/.venv/lib/python3.10/site-packages/juliacall/__init__.py", line 228 in <module>
  File "<frozen importlib._bootstrap>", line 241 in _call_with_frames_removed
  File "<frozen importlib._bootstrap_external>", line 883 in exec_module
  File "<frozen importlib._bootstrap>", line 688 in _load_unlocked
  File "<frozen importlib._bootstrap>", line 1006 in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 1027 in _find_and_load
  File "<ipython-input-2-c645ed7a88d4>", line 1 in <module>
  File "/Users/mcranmer/.pyenv/versions/3.10.10/lib/python3.10/site-packages/IPython/core/interactiveshell.py", line 3508 in run_code
  File "/Users/mcranmer/.pyenv/versions/3.10.10/lib/python3.10/site-packages/IPython/core/interactiveshell.py", line 3448 in run_ast_nodes
  File "/Users/mcranmer/.pyenv/versions/3.10.10/lib/python3.10/site-packages/IPython/core/interactiveshell.py", line 3269 in run_cell_async
  File "/Users/mcranmer/.pyenv/versions/3.10.10/lib/python3.10/site-packages/IPython/core/async_helpers.py", line 129 in _pseudo_sync_runner
  File "/Users/mcranmer/.pyenv/versions/3.10.10/lib/python3.10/site-packages/IPython/core/interactiveshell.py", line 3064 in _run_cell
  File "/Users/mcranmer/.pyenv/versions/3.10.10/lib/python3.10/site-packages/IPython/core/interactiveshell.py", line 3009 in run_cell
  File "/Users/mcranmer/.pyenv/versions/3.10.10/lib/python3.10/site-packages/IPython/terminal/interactiveshell.py", line 881 in interact
  File "/Users/mcranmer/.pyenv/versions/3.10.10/lib/python3.10/site-packages/IPython/terminal/interactiveshell.py", line 888 in mainloop
  File "/Users/mcranmer/.pyenv/versions/3.10.10/lib/python3.10/site-packages/IPython/terminal/ipapp.py", line 318 in start
  File "/Users/mcranmer/.pyenv/versions/3.10.10/lib/python3.10/site-packages/traitlets/config/application.py", line 1043 in launch_instance
  File "/Users/mcranmer/.pyenv/versions/3.10.10/lib/python3.10/site-packages/IPython/__init__.py", line 129 in start_ipython
  File "/Users/mcranmer/.pyenv/versions/3.10.10/bin/ipython", line 8 in <module>

[64935] signal (6): Abort trap: 6
in expression starting at none:2
__pthread_kill at /usr/lib/system/libsystem_kernel.dylib (unknown line)
Allocations: 596578 (Pool: 595526; Big: 1052); GC: 1
[1]    64935 abort      ipython

This will not be an issue if everyone moves to juliacall, but the transition period where some packages use PyCall (like PyPlot) and some packages use PythonCall may cause some of these issues.

Maybe there is a safety switch to get around this issue temporarily.

cjdoris commented 9 months ago

Yeah we should at least check for this and issue a better error message (like we do in PythonCall).

MilesCranmer commented 9 months ago

Maybe we could do it similarly to how PyTorch is handled

https://github.com/JuliaPy/PythonCall.jl/blob/13f596d6a7d60ef7bfcee2d538cd895f59826d95/pysrc/juliacall/__init__.py#L59-L64

But not sure if it needs to also be checked on the PythonCall.jl/PyCall.jl side?