search

JuliaGPU / ArrayFire.jl

Julia wrapper for the ArrayFire library
Other
204 stars 37 forks source link

Segmentation Fault #166

Open jgbos opened 7 years ago

jgbos commented 7 years ago

I'm having a segmentation fault that I'm having a hard time producing a minimal example for. I have zero issues with my code when I do not load ArrayFire, but once loaded I will have seg faults for doing operations like permutedims and getindex. Below I can get my code consistently fail running using the Interpolations package, but my code is doing a too much to explain here. It seems like it might be related to indexing into memory that no longer exists, but I cannot prove this and not sure how to begin debugging.

ArrayFire v3.5.1 (CUDA, 64-bit Linux, build 0a675e8)
Platform: CUDA Toolkit 8, Driver: 384.69
[0] Tesla K20c, 4743 MB, CUDA Compute 3.5
-1- Quadro K2200, 4042 MB, CUDA Compute 5.0

signal (11): Segmentation fault
while loading /home/ju17693/projects/ASDF/test.jl, in expression starting on line 67
unknown function (ip: 0x7f1a4a85c2a2)
collect at ./array.jl:441
unknown function (ip: 0x7f194a85c182)
jl_call_fptr_internal at /home/centos/buildbot/slave/package_tarball64/build/src/julia_internal.h:339 [inlined]
jl_call_method_internal at /home/centos/buildbot/slave/package_tarball64/build/src/julia_internal.h:358 [inlined]
jl_apply_generic at /home/centos/buildbot/slave/package_tarball64/build/src/gf.c:1933
index_gen at /home/ju17693/.julia/v0.6/Interpolations/src/gridded/linear.jl:40
jl_call_fptr_internal at /home/centos/buildbot/slave/package_tarball64/build/src/julia_internal.h:339 [inlined]
jl_call_method_internal at /home/centos/buildbot/slave/package_tarball64/build/src/julia_internal.h:358 [inlined]
jl_apply_generic at /home/centos/buildbot/slave/package_tarball64/build/src/gf.c:1933
jl_apply at /home/centos/buildbot/slave/package_tarball64/build/src/julia.h:1424 [inlined]
jl_f__apply at /home/centos/buildbot/slave/package_tarball64/build/src/builtins.c:426
index_gen at /home/ju17693/.julia/v0.6/Interpolations/src/b-splines/indexing.jl:28
jl_call_fptr_internal at /home/centos/buildbot/slave/package_tarball64/build/src/julia_internal.h:339 [inlined]
jl_call_method_internal at /home/centos/buildbot/slave/package_tarball64/build/src/julia_internal.h:358 [inlined]
jl_apply_generic at /home/centos/buildbot/slave/package_tarball64/build/src/gf.c:1933
jl_apply at /home/centos/buildbot/slave/package_tarball64/build/src/julia.h:1424 [inlined]
jl_f__apply at /home/centos/buildbot/slave/package_tarball64/build/src/builtins.c:426
index_gen at /home/ju17693/.julia/v0.6/Interpolations/src/gridded/linear.jl:38
jl_call_fptr_internal at /home/centos/buildbot/slave/package_tarball64/build/src/julia_internal.h:339 [inlined]
jl_call_method_internal at /home/centos/buildbot/slave/package_tarball64/build/src/julia_internal.h:358 [inlined]
jl_apply_generic at /home/centos/buildbot/slave/package_tarball64/build/src/gf.c:1933
jl_apply at /home/centos/buildbot/slave/package_tarball64/build/src/julia.h:1424 [inlined]
jl_f__apply at /home/centos/buildbot/slave/package_tarball64/build/src/builtins.c:426
index_gen at /home/ju17693/.julia/v0.6/Interpolations/src/b-splines/indexing.jl:28
jl_call_fptr_internal at /home/centos/buildbot/slave/package_tarball64/build/src/julia_internal.h:339 [inlined]
jl_call_method_internal at /home/centos/buildbot/slave/package_tarball64/build/src/julia_internal.h:358 [inlined]
jl_apply_generic at /home/centos/buildbot/slave/package_tarball64/build/src/gf.c:1933
jl_apply at /home/centos/buildbot/slave/package_tarball64/build/src/julia.h:1424 [inlined]
jl_f__apply at /home/centos/buildbot/slave/package_tarball64/build/src/builtins.c:426
index_gen at /home/ju17693/.julia/v0.6/Interpolations/src/gridded/linear.jl:38
index_gen at /home/ju17693/.julia/v0.6/Interpolations/src/gridded/linear.jl:38
getindex_impl at /home/ju17693/.julia/v0.6/Interpolations/src/gridded/indexing.jl:15
unknown function (ip: 0x7f194a85a852)
jl_call_fptr_internal at /home/centos/buildbot/slave/package_tarball64/build/src/julia_internal.h:339 [inlined]
jl_call_method_internal at /home/centos/buildbot/slave/package_tarball64/build/src/julia_internal.h:358 [inlined]
jl_apply_generic at /home/centos/buildbot/slave/package_tarball64/build/src/gf.c:1933
getindex at /home/ju17693/.julia/v0.6/Interpolations/src/gridded/indexing.jl:30
jl_call_staged at /home/centos/buildbot/slave/package_tarball64/build/src/method.c:263 [inlined]
jl_code_for_staged at /home/centos/buildbot/slave/package_tarball64/build/src/method.c:315
typeinf_frame at ./inference.jl:2476
typeinf_code at ./inference.jl:2564
unknown function (ip: 0x7f1a38fa96ad)
jl_call_fptr_internal at /home/centos/buildbot/slave/package_tarball64/build/src/julia_internal.h:339 [inlined]
jl_call_method_internal at /home/centos/buildbot/slave/package_tarball64/build/src/julia_internal.h:358 [inlined]
jl_apply_generic at /home/centos/buildbot/slave/package_tarball64/build/src/gf.c:1933
typeinf_ext at ./inference.jl:2603
unknown function (ip: 0x7f1a38f88782)
jl_call_fptr_internal at /home/centos/buildbot/slave/package_tarball64/build/src/julia_internal.h:339 [inlined]
jl_call_method_internal at /home/centos/buildbot/slave/package_tarball64/build/src/julia_internal.h:358 [inlined]
jl_apply_generic at /home/centos/buildbot/slave/package_tarball64/build/src/gf.c:1933
jl_apply at /home/centos/buildbot/slave/package_tarball64/build/src/julia.h:1424 [inlined]
jl_apply_with_saved_exception_state at /home/centos/buildbot/slave/package_tarball64/build/src/rtutils.c:257
jl_type_infer at /home/centos/buildbot/slave/package_tarball64/build/src/gf.c:269
jl_compile_for_dispatch at /home/centos/buildbot/slave/package_tarball64/build/src/gf.c:1668
jl_compile_method_internal at /home/centos/buildbot/slave/package_tarball64/build/src/julia_internal.h:307 [inlined]
jl_call_method_internal at /home/centos/buildbot/slave/package_tarball64/build/src/julia_internal.h:354 [inlined]
jl_apply_generic at /home/centos/buildbot/slave/package_tarball64/build/src/gf.c:1933
#segment_image#40 at /home/ju17693/projects/ASDF/ASDF/src/segment.jl:32
unknown function (ip: 0x7f194a84f20d)
jl_call_fptr_internal at /home/centos/buildbot/slave/package_tarball64/build/src/julia_internal.h:339 [inlined]
jl_call_method_internal at /home/centos/buildbot/slave/package_tarball64/build/src/julia_internal.h:358 [inlined]
jl_apply_generic at /home/centos/buildbot/slave/package_tarball64/build/src/gf.c:1933
#segment_image at ./<missing>:0
unknown function (ip: 0x7f194a84e3d6)
jl_call_fptr_internal at /home/centos/buildbot/slave/package_tarball64/build/src/julia_internal.h:339 [inlined]
jl_call_method_internal at /home/centos/buildbot/slave/package_tarball64/build/src/julia_internal.h:358 [inlined]
jl_apply_generic at /home/centos/buildbot/slave/package_tarball64/build/src/gf.c:1933
jl_apply at /home/centos/buildbot/slave/package_tarball64/build/src/julia.h:1424 [inlined]
jl_f__apply at /home/centos/buildbot/slave/package_tarball64/build/src/builtins.c:426
#segmentmap#48 at /home/ju17693/projects/ASDF/ASDFPlots/src/segment.jl:2
jl_call_fptr_internal at /home/centos/buildbot/slave/package_tarball64/build/src/julia_internal.h:339 [inlined]
jl_call_method_internal at /home/centos/buildbot/slave/package_tarball64/build/src/julia_internal.h:358 [inlined]
jl_apply_generic at /home/centos/buildbot/slave/package_tarball64/build/src/gf.c:1933
jl_apply at /home/centos/buildbot/slave/package_tarball64/build/src/julia.h:1424 [inlined]
jl_f__apply at /home/centos/buildbot/slave/package_tarball64/build/src/builtins.c:426
#segmentmap at ./<missing>:0
jl_call_fptr_internal at /home/centos/buildbot/slave/package_tarball64/build/src/julia_internal.h:339 [inlined]
jl_call_method_internal at /home/centos/buildbot/slave/package_tarball64/build/src/julia_internal.h:358 [inlined]
jl_apply_generic at /home/centos/buildbot/slave/package_tarball64/build/src/gf.c:1933
do_call at /home/centos/buildbot/slave/package_tarball64/build/src/interpreter.c:75
eval at /home/centos/buildbot/slave/package_tarball64/build/src/interpreter.c:242
jl_interpret_toplevel_expr at /home/centos/buildbot/slave/package_tarball64/build/src/interpreter.c:34
jl_toplevel_eval_flex at /home/centos/buildbot/slave/package_tarball64/build/src/toplevel.c:577
jl_parse_eval_all at /home/centos/buildbot/slave/package_tarball64/build/src/ast.c:873
jl_load at /home/centos/buildbot/slave/package_tarball64/build/src/toplevel.c:616
include_from_node1 at ./loading.jl:569
unknown function (ip: 0x7f1a39109b1b)
jl_call_fptr_internal at /home/centos/buildbot/slave/package_tarball64/build/src/julia_internal.h:339 [inlined]
jl_call_method_internal at /home/centos/buildbot/slave/package_tarball64/build/src/julia_internal.h:358 [inlined]
jl_apply_generic at /home/centos/buildbot/slave/package_tarball64/build/src/gf.c:1933
include at ./sysimg.jl:14
unknown function (ip: 0x7f1a38fae86b)
jl_call_fptr_internal at /home/centos/buildbot/slave/package_tarball64/build/src/julia_internal.h:339 [inlined]
jl_call_method_internal at /home/centos/buildbot/slave/package_tarball64/build/src/julia_internal.h:358 [inlined]
jl_apply_generic at /home/centos/buildbot/slave/package_tarball64/build/src/gf.c:1933
process_options at ./client.jl:305
_start at ./client.jl:371
unknown function (ip: 0x7f1a391156e8)
jl_call_fptr_internal at /home/centos/buildbot/slave/package_tarball64/build/src/julia_internal.h:339 [inlined]
jl_call_method_internal at /home/centos/buildbot/slave/package_tarball64/build/src/julia_internal.h:358 [inlined]
jl_apply_generic at /home/centos/buildbot/slave/package_tarball64/build/src/gf.c:1933
jl_apply at /home/centos/buildbot/slave/package_tarball64/build/ui/../src/julia.h:1424 [inlined]
true_main at /home/centos/buildbot/slave/package_tarball64/build/ui/repl.c:127
main at /home/centos/buildbot/slave/package_tarball64/build/ui/repl.c:264
__libc_start_main at /lib64/libc.so.6 (unknown line)
unknown function (ip: 0x40170c)
Allocations: 19832558 (Pool: 19746302; Big: 86256); GC: 175
Segmentation fault (core dumped)
ghost commented 7 years ago

What it is the function that is calling collect? Indexing is pretty inefficient with GPUs in general, but it this specific case it seems that there's some generic Julia function that is not implemented is being processed element by element as AbstractArray. To debug it you can remove two instances of "<: AbstractArray{T,N}" from ArrayFire/src/array.jl and see where things fail. (see https://github.com/JuliaComputing/ArrayFire.jl/issues/162)

jgbos commented 7 years ago

I'll take a look, but just realized I didn't make clear that the function that crashed wasn't using a GPU array. I had already extracted the data from the GPU. If I run the code without using GPU (not loading ArrayFire at all) I never get the segmentation fault. This is what is baffling me and making it hard to find a minimal reproducible example.

hayatoikoma commented 7 years ago

I also have the same issue when I use Interpolations.jl with ArrayFire and have not been able to make a minimal example for it. Since I have not thought that it was an issue of ArrayFire, I have never tried it without loading ArrayFire.

Might it be better to report this issue to Interpolations.jl as the array is not AFArray? It would be really helpful for me as well if this issue can be resolved.

dsweber2 commented 6 years ago

I'll say that I've had this issue in the past as well, and that the folks over at interpolations aren't really sure what to do about it either. My discussion of the issue is https://github.com/JuliaMath/Interpolations.jl/issues/187 . It didn't go away post 0.6.2, unfortunately.

I just now had a similar issue when not using Interpolations but some code I've written instead. I was able to make the issue go away by not running in-line in atom, which suggests that there's an interaction with atom.jl. But it's certainly an interruption to workflow to have to copy-paste lines of code to the terminal..