search

JuliaInterop / JavaCall.jl

Call Java from Julia
http://juliainterop.github.io/JavaCall.jl
Other
118 stars 53 forks source link

Segmentation fault on JavaCall.init #133

Closed rssdev10 closed 3 years ago

rssdev10 commented 3 years ago

Getting a stable error on:

export JULIA_COPY_STACKS=1 && julia
julia> using JavaCall
julia> JavaCall.init()
signal (11): Segmentation fault: 11
in expression starting at REPL[4]:1
unknown function (ip: 0x14396d62d)
Allocations: 6354888 (Pool: 6352794; Big: 2094); GC: 7

About the system:

julia> versioninfo()
Julia Version 1.5.2
Commit 539f3ce943 (2020-09-23 23:17 UTC)
Platform Info:
  OS: macOS (x86_64-apple-darwin18.7.0)
  CPU: Intel(R) Core(TM) i7-5557U CPU @ 3.10GHz
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-9.0.1 (ORCJIT, broadwell)
Environment:
  JULIA_COPY_STACKS = 1

MacOS 10.15.7 
JAVA_HOME=/Library/Java/JavaVirtualMachines/openjdk-14.0.2.jdk/Contents/Home

Debug version of Julia doesn't help:

> gdb julia-debug
GNU gdb (GDB) 9.2
Copyright (C) 2020 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Type "show copying" and "show warranty" for details.
This GDB was configured as "x86_64-apple-darwin19.5.0".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
    <http://www.gnu.org/software/gdb/documentation/>.
For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from julia-debug...
(gdb) r
Starting program: ~/tmp/julia/julia/usr/bin/julia-debug 
[New Thread 0xf03 of process 38426]
[New Thread 0x1603 of process 38426]
warning: unhandled dyld version (16)
               _
   _       _ _(_)_     |  Documentation: https://docs.julialang.org
  (_)     | (_) (_)    |
   _ _   _| |_  __ _   |  Type "?" for help, "]?" for Pkg help.
  | | | | | | |/ _` |  |
  | | |_| | | | (_| |  |  Version 1.5.2 (2020-09-23)
 _/ |\__'_|_|_|\__'_|  |  
|__/                   |
julia> using JavaCall
julia> JavaCall.init()
[New Thread 0x1507 of process 38426]
[New Thread 0x1703 of process 38426]
[New Thread 0x1803 of process 38426]
[New Thread 0x1903 of process 38426]
signal (11): Segmentation fault: 11
in expression starting at REPL[2]:1
unknown function (ip: 0x12bc2b62d)
Allocations: 4038501 (Pool: 4037321; Big: 1180); GC: 5
Thread 2 received signal SIGSEGV, Segmentation fault.
0x000000012bc2b62d in ?? ()
(gdb) bt
#0  0x000000012bc2b62d in ?? ()
#1  0x0000000000000246 in ?? ()
#2  0x000000012bc2b380 in ?? ()
#3  0x0000000123033200 in ?? ()
#4  0x00007ffeefbfaec0 in ?? ()
#5  0x0000000122e4c29d in ?? ()
#6  0x0000000000000006 in ?? ()
#7  0x0000000000000000 in ?? ()
(gdb)

At the same time, activation with --handle-signals=no allows to run Java:

julia --handle-signals=no --project=.
               _
   _       _ _(_)_     |  Documentation: https://docs.julialang.org
  (_)     | (_) (_)    |
   _ _   _| |_  __ _   |  Type "?" for help, "]?" for Pkg help.
  | | | | | | |/ _` |  |
  | | |_| | | | (_| |  |  Version 1.5.2 (2020-09-23)
 _/ |\__'_|_|_|\__'_|  |  Official https://julialang.org/ release
|__/                   |

julia> using JavaCall

julia> JavaCall.init()

julia> jlm = @jimport java.lang.Math
JavaObject{Symbol("java.lang.Math")}

julia> jcall(jlm, "sin", jdouble, (jdouble,), pi/2)
1.0
aviks commented 3 years ago

Hi @rssdev10

When you get the signal (11): Segmentation fault: 11 message, do you get the julia> prompt back after that, or does Julia exit to the shell?

rssdev10 commented 3 years ago

Checked again. After the segmentation fault the REPL is still active. And moreover, Java is activated.....

julia> using JavaCall
julia> JavaCall.init()

signal (11): Segmentation fault: 11
in expression starting at REPL[2]:1
unknown function (ip: 0x13080762d)
Allocations: 3206990 (Pool: 3205856; Big: 1134); GC: 4

julia> JavaCall.init()
ERROR: JavaCall.JavaCallError("JVM already initialised")
Stacktrace:
 [1] assertnotloaded at ~/.julia/packages/JavaCall/aVXyt/src/jvm.jl:242 [inlined]
 [2] _init(::OrderedCollections.OrderedSet{String}) at ~/.julia/packages/JavaCall/aVXyt/src/jvm.jl:284
 [3] init() at ~/.julia/packages/JavaCall/aVXyt/src/jvm.jl:272
 [4] top-level scope at REPL[2]:1

julia>

julia> jlm = @jimport java.lang.Math
JavaObject{Symbol("java.lang.Math")}

julia> jcall(jlm, "sin", jdouble, (jdouble,), pi/2)
1.0
aviks commented 3 years ago

Ok, so this is expected. As I mentioned on slack, the segfault that you see on the Mac is expected. This segfault is thrown and caught inside the JVM, but since Julia also installs a signal handler, it is reported. Since it's caught inside the JVM, it is harmless. The JVM uses segfaults as a performance optimization.

All of this is documented in our FAQs https://juliainterop.github.io/JavaCall.jl/faq.html

mkitti commented 3 years ago

Our CI tests do show a segmentation fault as well, but also show that the JavaCall tests do pass. https://travis-ci.com/github/mkitti/JavaCall.jl/jobs/373026648

Notably Mac is the only platform where this signal handling does get intercepted by Julia. It is not intercepted and reported by Linux or Windows. I do wonder if this means some parts of the JVM are not able to function correctly, particularly bounds checking and garbage collection.

The lingering question is though is if this is related to Taro.jl having problems loading Tika on Mac OS.

mkitti commented 3 years ago

Just to further the data collection for posterity, could you document here several things we discussed on Slack:

  1. The results of ]test JavaCall
  2. The results of doing the above with julia --handle-signals=no
  3. Also, do you see the same messages when you when you just run julia -e "using JavaCall; JavaCall.init()" 3a. Also do you see the same when running a script using JavaCall

It would be good to reference this as a potential issue in case this is actually indicative of a deeper bug, which might be in Julia itself.

rssdev10 commented 3 years ago

I also found that behavior of Julia is different in case of -e and REPL run:

> julia -e "using JavaCall; JavaCall.init();jlm = @jimport java.lang.Math; jcall(jlm, "sin", jdouble, (jdouble,), pi/2)"

signal (11): Segmentation fault: 11
in expression starting at none:1
unknown function (ip: 0x135cf662d)
Allocations: 1706585 (Pool: 1705879; Big: 706); GC: 1
ERROR: MethodError: no method matching jcall(::Type{JavaObject{Symbol("java.lang.Math")}}, ::typeof(sin), ::Type{Float64}, ::Tuple{DataType}, ::Float64)
Closest candidates are:
  jcall(::Type{JavaObject{T}}, ::AbstractString, ::Type, ::Tuple, ::Any...) where T at /Users/rss/.julia/packages/JavaCall/aVXyt/src/core.jl:217
  jcall(::JavaObject, ::AbstractString, ::Type, ::Tuple, ::Any...) at /Users/rss/.julia/packages/JavaCall/aVXyt/src/core.jl:227
Stacktrace:
 [1] top-level scope at none:1

vs

> julia
               _
   _       _ _(_)_     |  Documentation: https://docs.julialang.org
  (_)     | (_) (_)    |
   _ _   _| |_  __ _   |  Type "?" for help, "]?" for Pkg help.
  | | | | | | |/ _` |  |
  | | |_| | | | (_| |  |  Version 1.5.2 (2020-09-23)
 _/ |\__'_|_|_|\__'_|  |  Official https://julialang.org/ release
|__/                   |

julia> using JavaCall; JavaCall.init();jlm = @jimport java.lang.Math; jcall(jlm, "sin", jdouble, (jdouble,), pi/2)

signal (11): Segmentation fault: 11
in expression starting at REPL[1]:1
unknown function (ip: 0x12fbb762d)
Allocations: 3227104 (Pool: 3225920; Big: 1184); GC: 3
1.0
mkitti commented 3 years ago

Umm... you might need to escape the quotes when passing "sin" in. It would probably be easier to just make a script:

javacall_check.jl:

using JavaCall;
JavaCall.init();
jlm = @jimport java.lang.Math;

sin90 = jcall(jlm, "sin", jdouble, (jdouble,), pi/2)
println(sin90)

sin0 = jcall(jlm, "sin", jdouble, (jdouble,), 0)
println(sin0)

sin180 = jcall(jlm, "sin", jdouble, (jdouble,), pi)
println(sin180)

JULIA_COPY_STACKS=1 julia javacall_check.jl

rssdev10 commented 3 years ago

yes, you are right:

bash-3.2$ JULIA_COPY_STACKS=1 julia javacall_check.jl

signal (11): Segmentation fault: 11
in expression starting at /Users/rss/projects/tmp/julia/julia/usr/javacall_check.jl:2
unknown function (ip: 0x12841562d)
Allocations: 1706573 (Pool: 1705868; Big: 705); GC: 2
1.0
0.0
1.2246467991473532e-16

and

julia -e 'using JavaCall; JavaCall.init();jlm = @jimport java.lang.Math; res=jcall(jlm, "sin", jdouble, (jdouble,), pi/2); println(res)'

signal (11): Segmentation fault: 11
in expression starting at none:1
unknown function (ip: 0x126d5262d)
Allocations: 1706634 (Pool: 1705929; Big: 705); GC: 2
1.0
mkitti commented 3 years ago

Thanks for the result. This tells me that the signal handling is not dependent on the REPL or use of @async and JavaCall is functioning on some level.

mkitti commented 3 years ago

By the way, I also setup CI for Taro.jl and get the same result on Mac: https://travis-ci.com/github/mkitti/Taro.jl/jobs/407109720

rssdev10 commented 3 years ago

In your case, there is an issue with 'JAVAPATH' or java as is - [ Info: JavaCall could not determine javapath from ``which java``

pkg> test Taro
    Testing Taro
Status `/private/var/folders/n_/h8tm56xd4q9_41rd7knts2r80000gn/T/jl_2u6Vdl/Project.toml`
  [a93c6f00] DataFrames v0.21.8
  [494afd89] JavaCall v0.7.6
  [bd369af6] Tables v1.1.0
  [61d0e4fa] Taro v0.8.2
  [ade2ca70] Dates
  [8dfed614] Test
Status `/private/var/folders/n_/h8tm56xd4q9_41rd7knts2r80000gn/T/jl_2u6Vdl/Manifest.toml`
  [324d7699] CategoricalArrays v0.8.3
  [34da2185] Compat v3.21.0
  [9a962f9c] DataAPI v1.3.0
  [a93c6f00] DataFrames v0.21.8
  [864edb3b] DataStructures v0.18.7
  [e2d170a0] DataValueInterfaces v1.0.0
  [41ab1584] InvertedIndices v1.0.0
  [82899510] IteratorInterfaceExtensions v1.0.0
  [682c06a0] JSON v0.21.1
  [494afd89] JavaCall v0.7.6
  [e1d29d7a] Missings v0.4.4
  [bac558e1] OrderedCollections v1.3.1
  [69de0a69] Parsers v1.0.11
  [2dfb63ee] PooledArrays v0.5.3
  [189a3867] Reexport v0.2.0
  [a2af1166] SortingAlgorithms v0.3.1
  [856f2bd8] StructTypes v1.1.0
  [3783bdb8] TableTraits v1.0.0
  [bd369af6] Tables v1.1.0
  [61d0e4fa] Taro v0.8.2
  [1b915085] WinReg v0.3.1
  [2a0f44e3] Base64
  [ade2ca70] Dates
  [8bb1440f] DelimitedFiles
  [8ba89e20] Distributed
  [9fa8497b] Future
  [b77e0a4c] InteractiveUtils
  [76f85450] LibGit2
  [8f399da3] Libdl
  [37e2e46d] LinearAlgebra
  [56ddb016] Logging
  [d6f4376e] Markdown
  [a63ad114] Mmap
  [44cfe95a] Pkg
  [de0858da] Printf
  [3fa0cd96] REPL
  [9a3f8284] Random
  [ea8e919c] SHA
  [9e88b42a] Serialization
  [1a1011a3] SharedArrays
  [6462fe0b] Sockets
  [2f01184e] SparseArrays
  [10745b16] Statistics
  [8dfed614] Test
  [cf7118a7] UUIDs
  [4ec0a83e] Unicode

signal (11): Segmentation fault: 11
in expression starting at ~/.julia/packages/Taro/lMmIa/test/runtests.jl:12
unknown function (ip: 0x12c65e62d)
Allocations: 5714818 (Pool: 5712467; Big: 2351); GC: 6
Oct 26, 2020 9:43:41 AM org.apache.tika.config.InitializableProblemHandler$3 handleInitializableProblem
WARNING: J2KImageReader not loaded. JPEG2000 files will not be processed.
See https://pdfbox.apache.org/2.0/dependencies.html#jai-image-io
for optional dependencies.

Oct 26, 2020 9:43:41 AM org.apache.tika.config.InitializableProblemHandler$3 handleInitializableProblem
WARNING: org.xerial's sqlite-jdbc is not loaded.
Please provide the jar on your classpath to parse sqlite files.
See tika-parsers/pom.xml for the correct version.

signal (11): Segmentation fault: 11
in expression starting at ~/.julia/packages/Taro/lMmIa/test/runtests.jl:14
unknown function (ip: 0x13416dc68)
Allocations: 8291981 (Pool: 8289228; Big: 2753); GC: 9

signal (11): Segmentation fault: 11
in expression starting at ~/.julia/packages/Taro/lMmIa/test/runtests.jl:14
unknown function (ip: 0x134187225)
Allocations: 8291981 (Pool: 8289228; Big: 2753); GC: 9
No Java runtime present, requesting install.
ERROR: Package Taro errored during testing (exit code: 97)
mkitti commented 3 years ago

JavaCall tries a few different tricks to locate java. which java is only one of them. You're right though. It would be better if we specified JAVA_HOME

https://github.com/JuliaInterop/JavaCall.jl/blob/v0.7.6/src/jvm.jl#L26

I wonder where this goes wrong: https://github.com/JuliaInterop/JavaCall.jl/blob/bf72e80c0825dbc6128bbb409e08b0f16480dcc8/src/jvm.jl#L40-L46

rssdev10 commented 3 years ago 
julia> javapath = chomp(read(`which java`,String))
"/usr/bin/java"

julia> while(islink(javapath))
                           javapath = readlink(javapath)
                       end

julia> javapath
"/System/Library/Frameworks/JavaVM.framework/Versions/Current/Commands/java"

julia> javapath = dirname(javapath)
"/System/Library/Frameworks/JavaVM.framework/Versions/Current/Commands"

julia> javapath = match(r"(.*)(/jre)+(/bin)+",javapath)[1]
ERROR: MethodError: no method matching getindex(::Nothing, ::Int64)
Stacktrace:
 [1] top-level scope at REPL[9]:1

julia>

But on CentOS:

$ which java
/usr/bin/java 
$ readlink /usr/bin/java
/etc/alternatives/java
$ readlink /etc/alternatives/java
/usr/java/jdk1.8.0_31/jre/bin/java

or

$ which java
/usr/bin/java
$ readlink /usr/bin/java
/etc/alternatives/java
$ readlink /etc/alternatives/java
/usr/lib/jvm/java-11-openjdk-11.0.8.10-0.el8_2.x86_64/bin/java

BTW, the expression r"(.*)(/jre)+(/bin)+" contains a combination of (/jre) and (/bin) one or more times.... It should be rather something like (.*)(/jre)?/bin.

Also, for Mac - JAVA_HOME=/Library/Java/JavaVirtualMachines/openjdk-14.0.2.jdk/Contents/Home without jre