JuliaPy / PythonCall.jl

Python and Julia in harmony.
https://juliapy.github.io/PythonCall.jl/stable/
MIT License
720 stars 61 forks source link

`import juliacall` sometimes segfaults #384

Open LilithHafner opened 8 months ago

LilithHafner commented 8 months ago

Affects: JuliaCall

Describe the bug I sometimes get segfaults when importing julicall in combination with other packages.

x@x ~ % python -c 'import juliacall; import torch; from torch.autograd import Function'
x@x ~ % python -c 'import torch; from torch.autograd import Function; import juliacall'
x@x ~ % python -c 'import torch; from torch.autograd import Function; import juliacall'
zsh: segmentation fault  python3 -c 
x@x ~ % python -c 'import torch; from torch.autograd import Function; import juliacall'
zsh: segmentation fault  python3 -c 
x@x ~ % python -c 'import torch; from torch.autograd import Function; import juliacall'
zsh: segmentation fault  python3 -c 
x@x ~ % python -c 'import torch; from torch.autograd import Function; import juliacall'
zsh: segmentation fault  python3 -c 

Your system

x@x ~ % python --version
Python 3.11.2
x@x ~ % julia -e 'using InteractiveUtils; versioninfo()'
Julia Version 1.9.3
Commit bed2cd540a1 (2023-08-24 14:43 UTC)
Build Info:
  Official https://julialang.org/ release
Platform Info:
  OS: macOS (arm64-apple-darwin22.4.0)
  CPU: 8 × Apple M2
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-14.0.6 (ORCJIT, apple-m1)
  Threads: 1 on 4 virtual cores
Environment:
  JULIA_EDITOR = code
  JULIA_PYTHONCALL_EXE = /Library/Frameworks/Python.framework/Versions/3.11/bin/python3
x@x ~ % python -m pip list                              
Package                   Version
------------------------- ---------
absl-py                   2.0.0
accelerate                0.23.0
aiohttp                   3.8.4
aiosignal                 1.3.1
astunparse                1.6.3
async-timeout             4.0.2
attrs                     22.2.0
cachetools                5.3.1
certifi                   2022.12.7
cffi                      1.15.1
charset-normalizer        3.1.0
click                     8.1.3
codespell                 2.2.6
comtypes                  1.1.14
contourpy                 1.0.7
cycler                    0.11.0
diffeqpy                  2.1.0
diffusers                 0.21.1
Faker                     19.10.0
filelock                  3.12.4
fire                      0.5.0
flatbuffers               23.5.26
fonttools                 4.39.3
frozenlist                1.3.3
fsspec                    2023.9.0
gast                      0.4.0
google-auth               2.23.0
google-auth-oauthlib      1.0.0
google-pasta              0.2.0
grpcio                    1.58.0
gTTS                      2.3.1
h5py                      3.9.0
huggingface-hub           0.17.1
idna                      3.4
importlib-metadata        6.8.0
jill                      0.11.5
Jinja2                    3.1.2
jsonschema                4.19.0
jsonschema-specifications 2023.7.1
juliacall                 0.9.14
juliapkg                  0.1.10
keras                     2.13.1
kiwisolver                1.4.4
libclang                  16.0.6
llvmlite                  0.41.0
Markdown                  3.4.4
MarkupSafe                2.1.3
matplotlib                3.7.1
mecab-python3             1.0.6
mpmath                    1.3.0
multidict                 6.0.4
nanoid                    2.0.0
networkx                  3.1
numba                     0.58.0
numpy                     1.24.2
oauthlib                  3.2.2
openai                    0.27.2
opencv-python             4.7.0.72
opt-einsum                3.3.0
packaging                 23.1
Pillow                    9.5.0
pip                       23.2.1
playsound                 1.3.0
protobuf                  4.24.3
psutil                    5.9.5
pyasn1                    0.5.0
pyasn1-modules            0.3.0
pycparser                 2.21
pyobjc-framework-Vision   9.0.1
pyparsing                 3.0.9
python-dateutil           2.8.2
python-gnupg              0.5.1
pyttsx3                   2.91
PyYAML                    6.0.1
referencing               0.30.2
regex                     2023.8.8
requests                  2.28.2
requests-futures          1.0.1
requests-oauthlib         1.3.1
rlvoice                   1.0.0
rpds-py                   0.10.2
rsa                       4.9
safetensors               0.3.3
semantic-version          2.10.0
setuptools                65.5.0
six                       1.16.0
sounddevice               0.4.6
srt                       3.5.3
sympy                     1.12
tensorboard               2.13.0
tensorboard-data-server   0.7.1
tensorflow                2.13.0
tensorflow-estimator      2.13.0
tensorflow-macos          2.13.0
termcolor                 2.3.0
tokenizers                0.13.3
torch                     2.0.1
tqdm                      4.65.0
transformers              4.33.1
typing_extensions         4.5.0
urllib3                   1.26.15
vosk                      0.3.44
websockets                10.4
Werkzeug                  2.3.7
wget                      3.2
wheel                     0.41.2
wrapt                     1.15.0
yarl                      1.8.2
zipp                      3.16.2
x@x ~ % julia                                           
               _
   _       _ _(_)_     |  Documentation: https://docs.julialang.org
  (_)     | (_) (_)    |
   _ _   _| |_  __ _   |  Type "?" for help, "]?" for Pkg help.
  | | | | | | |/ _` |  |
  | | |_| | | | (_| |  |  Version 1.9.3 (2023-08-24)
 _/ |\__'_|_|_|\__'_|  |  Official https://julialang.org/ release
|__/                   |

julia> juliapkg.status()
ERROR: UndefVarError: `juliapkg` not defined
Stacktrace:
 [1] top-level scope
   @ REPL[1]:1

julia> 
x@x ~ % python -c 'import juliacall; juliacall.juliapkg.status()'
┌ Warning: attempting to remove probably stale pidfile
│   path = "/Users/x/.julia/logs/manifest_usage.toml.pid"
└ @ FileWatching.Pidfile ~/.julia/juliaup/julia-1.9.3+0.aarch64.apple.darwin14/share/julia/stdlib/v1.9/FileWatching/src/pidfile.jl:244
Traceback (most recent call last):
  File "<string>", line 1, in <module>
AttributeError: module 'juliacall' has no attribute 'juliapkg'
cjdoris commented 8 months ago

Yeah it's a known issue. See also #215.

cjdoris commented 8 months ago

That issue suggested printing a warning if torch is loaded before juliacall, which was never implemented.