search

tkf / ThreadsX.jl

Parallelized Base functions
MIT License
321 stars 10 forks source link

Segfault when using `ThreadsX.sum` with Zygote on ARM #192

Closed SebastianAment closed 2 years ago

SebastianAment commented 2 years ago

I just tried to parallelize a component of a model with ThreadsX.sum and subsequently differentiate through it with Zygote.gradient. This led to a segfault, but I am so far unable to give a MWE. I put the error message in the following. I am running Julia 1.7.1 natively on an M1 Pro (ARM).

signal (11): Segmentation fault: 11
in expression starting at /Users/sebastianament/Documents/SEA/GaussianProcessPhaseMapping/test/gppm.jl:56
ntuple at ./ntuple.jl:0
unknown function (ip: 0x117d153ef)
jl_apply_generic at /Applications/Julia-1.7.app/Contents/Resources/julia/lib/julia/libjulia-internal.1.7.dylib (unknown line)
getindex at ./range.jl:373
jfptr_getindex_24929 at /Applications/Julia-1.7.app/Contents/Resources/julia/lib/julia/sys.dylib (unknown line)
jl_apply_generic at /Applications/Julia-1.7.app/Contents/Resources/julia/lib/julia/libjulia-internal.1.7.dylib (unknown line)
#s3060#1216 at /Users/sebastianament/.julia/packages/Zygote/FPUm3/src/compiler/interface2.jl:13 [inlined]
#s3060#1216 at ./none:0
jl_apply_generic at /Applications/Julia-1.7.app/Contents/Resources/julia/lib/julia/libjulia-internal.1.7.dylib (unknown line)
GeneratedFunctionStub at ./boot.jl:580
jl_apply_generic at /Applications/Julia-1.7.app/Contents/Resources/julia/lib/julia/libjulia-internal.1.7.dylib (unknown line)
jl_call_staged at /Applications/Julia-1.7.app/Contents/Resources/julia/lib/julia/libjulia-internal.1.7.dylib (unknown line)
jl_code_for_staged at /Applications/Julia-1.7.app/Contents/Resources/julia/lib/julia/libjulia-internal.1.7.dylib (unknown line)
get_staged at ./compiler/utilities.jl:111
retrieve_code_info at ./compiler/utilities.jl:123 [inlined]
InferenceState at ./compiler/inferencestate.jl:234
typeinf_edge at ./compiler/typeinfer.jl:814 [inlined]
abstract_call_method at ./compiler/abstractinterpretation.jl:504
abstract_call_gf_by_type at ./compiler/abstractinterpretation.jl:105
abstract_call_known at ./compiler/abstractinterpretation.jl:1342
abstract_call at ./compiler/abstractinterpretation.jl:1397
abstract_call at ./compiler/abstractinterpretation.jl:1382
abstract_eval_statement at ./compiler/abstractinterpretation.jl:1534
typeinf_local at ./compiler/abstractinterpretation.jl:1918
typeinf_nocycle at ./compiler/abstractinterpretation.jl:2014
_typeinf at ./compiler/typeinfer.jl:226
typeinf at ./compiler/typeinfer.jl:209
typeinf_edge at ./compiler/typeinfer.jl:823 [inlined]
abstract_call_method at ./compiler/abstractinterpretation.jl:504
abstract_call_gf_by_type at ./compiler/abstractinterpretation.jl:105
abstract_call_known at ./compiler/abstractinterpretation.jl:1342
abstract_call at ./compiler/abstractinterpretation.jl:1397
abstract_call at ./compiler/abstractinterpretation.jl:1382
abstract_eval_statement at ./compiler/abstractinterpretation.jl:1534
typeinf_local at ./compiler/abstractinterpretation.jl:1918
typeinf_nocycle at ./compiler/abstractinterpretation.jl:2014
_typeinf at ./compiler/typeinfer.jl:226
typeinf at ./compiler/typeinfer.jl:209
typeinf_edge at ./compiler/typeinfer.jl:823 [inlined]
abstract_call_method at ./compiler/abstractinterpretation.jl:504
abstract_call_gf_by_type at ./compiler/abstractinterpretation.jl:105
abstract_call_known at ./compiler/abstractinterpretation.jl:1342
abstract_call at ./compiler/abstractinterpretation.jl:1397
abstract_call at ./compiler/abstractinterpretation.jl:1382
abstract_eval_statement at ./compiler/abstractinterpretation.jl:1534
typeinf_local at ./compiler/abstractinterpretation.jl:1918
typeinf_nocycle at ./compiler/abstractinterpretation.jl:2014
_typeinf at ./compiler/typeinfer.jl:226
typeinf at ./compiler/typeinfer.jl:209
typeinf_edge at ./compiler/typeinfer.jl:823 [inlined]
abstract_call_method at ./compiler/abstractinterpretation.jl:504
abstract_call_gf_by_type at ./compiler/abstractinterpretation.jl:105
abstract_call_known at ./compiler/abstractinterpretation.jl:1342
abstract_call at ./compiler/abstractinterpretation.jl:1397
abstract_call at ./compiler/abstractinterpretation.jl:1382
abstract_eval_statement at ./compiler/abstractinterpretation.jl:1534
typeinf_local at ./compiler/abstractinterpretation.jl:1918
typeinf_nocycle at ./compiler/abstractinterpretation.jl:2014
_typeinf at ./compiler/typeinfer.jl:226
typeinf at ./compiler/typeinfer.jl:209
typeinf_ext at ./compiler/typeinfer.jl:909
typeinf_ext_toplevel at ./compiler/typeinfer.jl:942
typeinf_ext_toplevel at ./compiler/typeinfer.jl:938
jfptr_typeinf_ext_toplevel_14075 at /Applications/Julia-1.7.app/Contents/Resources/julia/lib/julia/sys.dylib (unknown line)
jl_apply_generic at /Applications/Julia-1.7.app/Contents/Resources/julia/lib/julia/libjulia-internal.1.7.dylib (unknown line)
jl_type_infer at /Applications/Julia-1.7.app/Contents/Resources/julia/lib/julia/libjulia-internal.1.7.dylib (unknown line)
jl_generate_fptr at /Applications/Julia-1.7.app/Contents/Resources/julia/lib/julia/libjulia-internal.1.7.dylib (unknown line)
jl_compile_method_internal at /Applications/Julia-1.7.app/Contents/Resources/julia/lib/julia/libjulia-internal.1.7.dylib (unknown line)
jl_apply_generic at /Applications/Julia-1.7.app/Contents/Resources/julia/lib/julia/libjulia-internal.1.7.dylib (unknown line)
_pullback at /Users/sebastianament/.julia/packages/ThreadsX/smC2N/src/reduce.jl:19 [inlined]
_pullback at /Users/sebastianament/.julia/packages/Zygote/FPUm3/src/compiler/interface2.jl:0
_pullback at /Users/sebastianament/.julia/packages/ThreadsX/smC2N/src/reduce.jl:18 [inlined]
_pullback at /Users/sebastianament/.julia/packages/ThreadsX/smC2N/src/reduce.jl:48 [inlined]
_pullback at /Users/sebastianament/.julia/packages/ThreadsX/smC2N/src/reduce.jl:48 [inlined]
_pullback at /Users/sebastianament/.julia/packages/Zygote/FPUm3/src/compiler/interface2.jl:0
unknown function (ip: 0x117cd924f)
jl_apply_generic at /Applications/Julia-1.7.app/Contents/Resources/julia/lib/julia/libjulia-internal.1.7.dylib (unknown line)
_pullback at /Users/sebastianament/Documents/SEA/GaussianProcessPhaseMapping/src/library.jl:25 [inlined]
_pullback at /Users/sebastianament/.julia/packages/Zygote/FPUm3/src/compiler/interface2.jl:0
_pullback at /Users/sebastianament/Documents/SEA/GaussianProcessPhaseMapping/src/library.jl:35 [inlined]
_pullback at /Users/sebastianament/.julia/packages/Zygote/FPUm3/src/compiler/interface2.jl:0
unknown function (ip: 0x117cce5ef)
jl_apply_generic at /Applications/Julia-1.7.app/Contents/Resources/julia/lib/julia/libjulia-internal.1.7.dylib (unknown line)
_pullback at /Users/sebastianament/.julia/packages/Zygote/FPUm3/src/compiler/interface.jl:34
pullback at /Users/sebastianament/.julia/packages/Zygote/FPUm3/src/compiler/interface.jl:40
gradient at /Users/sebastianament/.julia/packages/Zygote/FPUm3/src/compiler/interface.jl:75
unknown function (ip: 0x117ccb5c3)
library_gradient at /Users/sebastianament/Documents/SEA/GaussianProcessPhaseMapping/src/library.jl:38 [inlined]
library_gradient at /Users/sebastianament/Documents/SEA/GaussianProcessPhaseMapping/src/library.jl:30
unknown function (ip: 0x117ccae73)
jl_apply_generic at /Applications/Julia-1.7.app/Contents/Resources/julia/lib/julia/libjulia-internal.1.7.dylib (unknown line)
top-level scope at ./timing.jl:220
jl_toplevel_eval_flex at /Applications/Julia-1.7.app/Contents/Resources/julia/lib/julia/libjulia-internal.1.7.dylib (unknown line)
jl_toplevel_eval_flex at /Applications/Julia-1.7.app/Contents/Resources/julia/lib/julia/libjulia-internal.1.7.dylib (unknown line)
jl_toplevel_eval_in at /Applications/Julia-1.7.app/Contents/Resources/julia/lib/julia/libjulia-internal.1.7.dylib (unknown line)
eval at ./boot.jl:373 [inlined]
include_string at ./loading.jl:1196
jl_apply_generic at /Applications/Julia-1.7.app/Contents/Resources/julia/lib/julia/libjulia-internal.1.7.dylib (unknown line)
_include at ./loading.jl:1253
include at ./client.jl:451
jl_apply_generic at /Applications/Julia-1.7.app/Contents/Resources/julia/lib/julia/libjulia-internal.1.7.dylib (unknown line)
do_call at /Applications/Julia-1.7.app/Contents/Resources/julia/lib/julia/libjulia-internal.1.7.dylib (unknown line)
eval_body at /Applications/Julia-1.7.app/Contents/Resources/julia/lib/julia/libjulia-internal.1.7.dylib (unknown line)
jl_interpret_toplevel_thunk at /Applications/Julia-1.7.app/Contents/Resources/julia/lib/julia/libjulia-internal.1.7.dylib (unknown line)
jl_toplevel_eval_flex at /Applications/Julia-1.7.app/Contents/Resources/julia/lib/julia/libjulia-internal.1.7.dylib (unknown line)
jl_toplevel_eval_flex at /Applications/Julia-1.7.app/Contents/Resources/julia/lib/julia/libjulia-internal.1.7.dylib (unknown line)
jl_toplevel_eval_flex at /Applications/Julia-1.7.app/Contents/Resources/julia/lib/julia/libjulia-internal.1.7.dylib (unknown line)
jl_toplevel_eval_in at /Applications/Julia-1.7.app/Contents/Resources/julia/lib/julia/libjulia-internal.1.7.dylib (unknown line)
eval at ./boot.jl:373 [inlined]
eval_user_input at /Users/sabae/src/julia/usr/share/julia/stdlib/v1.7/REPL/src/REPL.jl:150
repl_backend_loop at /Users/sabae/src/julia/usr/share/julia/stdlib/v1.7/REPL/src/REPL.jl:244
start_repl_backend at /Users/sabae/src/julia/usr/share/julia/stdlib/v1.7/REPL/src/REPL.jl:229
#run_repl#47 at /Users/sabae/src/julia/usr/share/julia/stdlib/v1.7/REPL/src/REPL.jl:362
run_repl at /Users/sabae/src/julia/usr/share/julia/stdlib/v1.7/REPL/src/REPL.jl:349
jl_apply_generic at /Applications/Julia-1.7.app/Contents/Resources/julia/lib/julia/libjulia-internal.1.7.dylib (unknown line)
#930 at ./client.jl:394
jfptr_YY.930_29470 at /Applications/Julia-1.7.app/Contents/Resources/julia/lib/julia/sys.dylib (unknown line)
jl_apply_generic at /Applications/Julia-1.7.app/Contents/Resources/julia/lib/julia/libjulia-internal.1.7.dylib (unknown line)
jl_f__call_latest at /Applications/Julia-1.7.app/Contents/Resources/julia/lib/julia/libjulia-internal.1.7.dylib (unknown line)
#invokelatest#2 at ./essentials.jl:716 [inlined]
invokelatest at ./essentials.jl:714 [inlined]
run_main_repl at ./client.jl:379
exec_options at ./client.jl:309
_start at ./client.jl:495
jl_sysimg_fvars_base at /Applications/Julia-1.7.app/Contents/Resources/julia/lib/julia/sys.dylib (unknown line)
jl_apply_generic at /Applications/Julia-1.7.app/Contents/Resources/julia/lib/julia/libjulia-internal.1.7.dylib (unknown line)
true_main at /Applications/Julia-1.7.app/Contents/Resources/julia/lib/julia/libjulia-internal.1.7.dylib (unknown line)
jl_repl_entrypoint at /Applications/Julia-1.7.app/Contents/Resources/julia/lib/julia/libjulia-internal.1.7.dylib (unknown line)
Allocations: 125899981 (Pool: 125856217; Big: 43764); GC: 103
rejuvyesh commented 2 years ago

For threaded computation on M1 pro, there are a bunch of fixes coming with Julia v1.8.

SebastianAment commented 2 years ago

Closing because it seems to be fixed on nightly.