search

JuliaIO / JLD.jl

Saving and loading julia variables while preserving native types
MIT License
278 stars 55 forks source link

Seg Fault when loading large Float16 Arrays in jld files. #330

Open ericrouviere opened 7 months ago

ericrouviere commented 7 months ago

I get a seq fault with the following code in Julia 1.10.0 (and 1.9.2) while using JLD v0.13.4. The error does not occur with Float64s, smaller Float16 vectors. This error does not occur in Julia 1.7.2 with v0.13.4.

julia> using JLD

julia> A = Float16.(rand(5280000));

julia> save("A.jld","1", A)

julia> load("A.jld","1")

[19886] signal (11.1): Segmentation fault
in expression starting at REPL[4]:1
unsafe_load at ./pointer.jl:119 [inlined]
unsafe_load at ./pointer.jl:119 [inlined]
_jlconvert_bits! at /home/erouviere/.julia/packages/JLD/GHujm/src/jld_types.jl:126 [inlined]
macro expansion at /home/erouviere/.julia/packages/JLD/GHujm/src/jld_types.jl:535 [inlined]
jlconvert! at /home/erouviere/.julia/packages/JLD/GHujm/src/jld_types.jl:546
unknown function (ip: 0x7f0d8b60e496)
_jl_invoke at /cache/build/builder-amdci4-6/julialang/julia-release-1-dot-10/src/gf.c:2894 [inlined]
ijl_apply_generic at /cache/build/builder-amdci4-6/julialang/julia-release-1-dot-10/src/gf.c:3076
read_vals_default at /home/erouviere/.julia/packages/JLD/GHujm/src/JLD.jl:478
read_vals at /home/erouviere/.julia/packages/JLD/GHujm/src/JLD.jl:455
unknown function (ip: 0x7f0d8b60d112)
_jl_invoke at /cache/build/builder-amdci4-6/julialang/julia-release-1-dot-10/src/gf.c:2894 [inlined]
ijl_apply_generic at /cache/build/builder-amdci4-6/julialang/julia-release-1-dot-10/src/gf.c:3076
read_array at /home/erouviere/.julia/packages/JLD/GHujm/src/JLD.jl:432
unknown function (ip: 0x7f0d8b5ffe5f)
_jl_invoke at /cache/build/builder-amdci4-6/julialang/julia-release-1-dot-10/src/gf.c:2894 [inlined]
ijl_apply_generic at /cache/build/builder-amdci4-6/julialang/julia-release-1-dot-10/src/gf.c:3076
read_array at /home/erouviere/.julia/packages/JLD/GHujm/src/JLD.jl:429
read at /home/erouviere/.julia/packages/JLD/GHujm/src/JLD.jl:388
read at /home/erouviere/.julia/packages/JLD/GHujm/src/JLD.jl:362
#48 at /home/erouviere/.julia/packages/JLD/GHujm/src/JLD.jl:1335 [inlined]
#jldopen#14 at /home/erouviere/.julia/packages/JLD/GHujm/src/JLD.jl:262
jldopen at /home/erouviere/.julia/packages/JLD/GHujm/src/JLD.jl:259 [inlined]
fileio_load at /home/erouviere/.julia/packages/JLD/GHujm/src/JLD.jl:1334
unknown function (ip: 0x7f0d8b5ff6e9)
_jl_invoke at /cache/build/builder-amdci4-6/julialang/julia-release-1-dot-10/src/gf.c:2894 [inlined]
ijl_apply_generic at /cache/build/builder-amdci4-6/julialang/julia-release-1-dot-10/src/gf.c:3076
jl_apply at /cache/build/builder-amdci4-6/julialang/julia-release-1-dot-10/src/julia.h:1982 [inlined]
jl_f__call_latest at /cache/build/builder-amdci4-6/julialang/julia-release-1-dot-10/src/builtins.c:812
#invokelatest#2 at ./essentials.jl:887 [inlined]
invokelatest at ./essentials.jl:884 [inlined]
#action#33 at /home/erouviere/.julia/packages/FileIO/jMf68/src/loadsave.jl:219
action at /home/erouviere/.julia/packages/FileIO/jMf68/src/loadsave.jl:196 [inlined]
#action#32 at /home/erouviere/.julia/packages/FileIO/jMf68/src/loadsave.jl:185 [inlined]
action at /home/erouviere/.julia/packages/FileIO/jMf68/src/loadsave.jl:185 [inlined]
#load#14 at /home/erouviere/.julia/packages/FileIO/jMf68/src/loadsave.jl:113
load at /home/erouviere/.julia/packages/FileIO/jMf68/src/loadsave.jl:109
unknown function (ip: 0x7f0d8b5fe0e9)
_jl_invoke at /cache/build/builder-amdci4-6/julialang/julia-release-1-dot-10/src/gf.c:2894 [inlined]
ijl_apply_generic at /cache/build/builder-amdci4-6/julialang/julia-release-1-dot-10/src/gf.c:3076
jl_apply at /cache/build/builder-amdci4-6/julialang/julia-release-1-dot-10/src/julia.h:1982 [inlined]
do_call at /cache/build/builder-amdci4-6/julialang/julia-release-1-dot-10/src/interpreter.c:126
eval_value at /cache/build/builder-amdci4-6/julialang/julia-release-1-dot-10/src/interpreter.c:223
eval_stmt_value at /cache/build/builder-amdci4-6/julialang/julia-release-1-dot-10/src/interpreter.c:174 [inlined]
eval_body at /cache/build/builder-amdci4-6/julialang/julia-release-1-dot-10/src/interpreter.c:617
jl_interpret_toplevel_thunk at /cache/build/builder-amdci4-6/julialang/julia-release-1-dot-10/src/interpreter.c:775
jl_toplevel_eval_flex at /cache/build/builder-amdci4-6/julialang/julia-release-1-dot-10/src/toplevel.c:934
jl_toplevel_eval_flex at /cache/build/builder-amdci4-6/julialang/julia-release-1-dot-10/src/toplevel.c:877
ijl_toplevel_eval_in at /cache/build/builder-amdci4-6/julialang/julia-release-1-dot-10/src/toplevel.c:985
eval at ./boot.jl:385 [inlined]
eval_user_input at /cache/build/builder-amdci4-6/julialang/julia-release-1-dot-10/usr/share/julia/stdlib/v1.10/REPL/src/REPL.jl:150
repl_backend_loop at /cache/build/builder-amdci4-6/julialang/julia-release-1-dot-10/usr/share/julia/stdlib/v1.10/REPL/src/REPL.jl:246
#start_repl_backend#46 at /cache/build/builder-amdci4-6/julialang/julia-release-1-dot-10/usr/share/julia/stdlib/v1.10/REPL/src/REPL.jl:231
start_repl_backend at /cache/build/builder-amdci4-6/julialang/julia-release-1-dot-10/usr/share/julia/stdlib/v1.10/REPL/src/REPL.jl:228
_jl_invoke at /cache/build/builder-amdci4-6/julialang/julia-release-1-dot-10/src/gf.c:2894 [inlined]
ijl_apply_generic at /cache/build/builder-amdci4-6/julialang/julia-release-1-dot-10/src/gf.c:3076
#run_repl#59 at /cache/build/builder-amdci4-6/julialang/julia-release-1-dot-10/usr/share/julia/stdlib/v1.10/REPL/src/REPL.jl:389
run_repl at /cache/build/builder-amdci4-6/julialang/julia-release-1-dot-10/usr/share/julia/stdlib/v1.10/REPL/src/REPL.jl:375
jfptr_run_repl_91689.1 at /home/erouviere/.julia/juliaup/julia-1.10.0+0.x64.linux.gnu/lib/julia/sys.so (unknown line)
_jl_invoke at /cache/build/builder-amdci4-6/julialang/julia-release-1-dot-10/src/gf.c:2894 [inlined]
ijl_apply_generic at /cache/build/builder-amdci4-6/julialang/julia-release-1-dot-10/src/gf.c:3076
#1013 at ./client.jl:432
jfptr_YY.1013_82677.1 at /home/erouviere/.julia/juliaup/julia-1.10.0+0.x64.linux.gnu/lib/julia/sys.so (unknown line)
_jl_invoke at /cache/build/builder-amdci4-6/julialang/julia-release-1-dot-10/src/gf.c:2894 [inlined]
ijl_apply_generic at /cache/build/builder-amdci4-6/julialang/julia-release-1-dot-10/src/gf.c:3076
jl_apply at /cache/build/builder-amdci4-6/julialang/julia-release-1-dot-10/src/julia.h:1982 [inlined]
jl_f__call_latest at /cache/build/builder-amdci4-6/julialang/julia-release-1-dot-10/src/builtins.c:812
#invokelatest#2 at ./essentials.jl:887 [inlined]
invokelatest at ./essentials.jl:884 [inlined]
run_main_repl at ./client.jl:416
exec_options at ./client.jl:333
_start at ./client.jl:552
jfptr__start_82703.1 at /home/erouviere/.julia/juliaup/julia-1.10.0+0.x64.linux.gnu/lib/julia/sys.so (unknown line)
_jl_invoke at /cache/build/builder-amdci4-6/julialang/julia-release-1-dot-10/src/gf.c:2894 [inlined]
ijl_apply_generic at /cache/build/builder-amdci4-6/julialang/julia-release-1-dot-10/src/gf.c:3076
jl_apply at /cache/build/builder-amdci4-6/julialang/julia-release-1-dot-10/src/julia.h:1982 [inlined]
true_main at /cache/build/builder-amdci4-6/julialang/julia-release-1-dot-10/src/jlapi.c:582
jl_repl_entrypoint at /cache/build/builder-amdci4-6/julialang/julia-release-1-dot-10/src/jlapi.c:731
main at /cache/build/builder-amdci4-6/julialang/julia-release-1-dot-10/cli/loader_exe.c:58
__libc_start_main at /lib/x86_64-linux-gnu/libc.so.6 (unknown line)
unknown function (ip: 0x4010b8)
Allocations: 27654673 (Pool: 27648233; Big: 6440); GC: 20
fatal: error thrown and no exception handler available.
MethodError(f=JLD.jlconvert!, args=(0x00007f0d46186098, Float16, JLD.JldFile(plain=HDF5.File(id=72057594037927937, filename="A.jld"), version=Base.VersionNumber(major=0x00000000, minor=0x00000001, patch=0x00000003, prerelease=(), build=()), toclose=true, writeheader=false, mmaparrays=false, compatible=false, compress=false, h5jltype=Base.Dict{Int64, Type}(slots=Array{UInt8, (16,)}[0x00, 0x00, 0x00, 0xca, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00], keys=Array{Int64, (16,)}[139696391571072, 0, 139696408612240, 4384, 139696391571088, 0, 139696258412064, 0, 139696326149664, 1, 139696326149664, 1, 139696326149664, 1, 139696326149664, 1], vals=Array{Type, (16,)}[
  #<null>,
  #<null>,
  #<null>,
  Float16,
  #<null>,
  #<null>,
  #<null>,
  #<null>,
  #<null>,
  #<null>,
  #<null>,
  #<null>,
  #<null>,
  #<null>,
  #<null>,
  #<null>], ndel=0, count=1, age=0x0000000000000001, idxfloor=4, maxprobe=0), jlh5type=Base.Dict{Type, JLD.JldDatatype}(slots=Array{UInt8, (16,)}[0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0xdb, 0x00, 0x00, 0x00, 0x00, 0x00], keys=Array{Type, (16,)}[
  #<null>,
  #<null>,
  #<null>,
  #<null>,
  #<null>,
  #<null>,
  #<null>,
  #<null>,
  #<null>,
  #<null>,
  Float16,
  #<null>,
  #<null>,
  #<null>,
  #<null>,
  #<null>], vals=Array{JLD.JldDatatype, (16,)}[JLD.JldDatatype(dtype=#<null>, index=0), JLD.JldDatatype(dtype=#<null>, index=0), JLD.JldDatatype(dtype=#<null>, index=0), JLD.JldDatatype(dtype=#<null>, index=0), JLD.JldDatatype(dtype=#<null>, index=0), JLD.JldDatatype(dtype=#<null>, index=0), JLD.JldDatatype(dtype=#<null>, index=0), JLD.JldDatatype(dtype=#<null>, index=0), JLD.JldDatatype(dtype=#<null>, index=0), JLD.JldDatatype(dtype=#<null>, index=0), JLD.JldDatatype(dtype=HDF5.Datatype(id=216172782113784254, toclose=true, file=HDF5.File(id=72057594037927937, filename="A.jld")), index=1), JLD.JldDatatype(dtype=#<null>, index=0), JLD.JldDatatype(dtype=#<null>, index=0), JLD.JldDatatype(dtype=#<null>, index=0), JLD.JldDatatype(dtype=#<null>, index=0), JLD.JldDatatype(dtype=#<null>, index=0)], ndel=0, count=1, age=0x0000000000000001, idxfloor=11, maxprobe=0), jlref=Base.Dict{HDF5.Reference, WeakRef}(slots=Array{UInt8, (16,)}[0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00], keys=Array{HDF5.Reference, (16,)}[HDF5.Reference(r=HDF5.API.hobj_ref_t(buf=0x00007f0d99cbc680)), HDF5.Reference(r=HDF5.API.hobj_ref_t(buf=0x0000000000000000)), HDF5.Reference(r=HDF5.API.hobj_ref_t(buf=0x00007f0d9acfcd90)), HDF5.Reference(r=HDF5.API.hobj_ref_t(buf=0x0000000000000000)), HDF5.Reference(r=HDF5.API.hobj_ref_t(buf=0x00007f0d99cbc690)), HDF5.Reference(r=HDF5.API.hobj_ref_t(buf=0x0000000000000000)), HDF5.Reference(r=HDF5.API.hobj_ref_t(buf=0x00007f0d91dbee20)), HDF5.Reference(r=HDF5.API.hobj_ref_t(buf=0x0000000000000000)), HDF5.Reference(r=HDF5.API.hobj_ref_t(buf=0x00007f0d95e58620)), HDF5.Reference(r=HDF5.API.hobj_ref_t(buf=0x0000000000000001)), HDF5.Reference(r=HDF5.API.hobj_ref_t(buf=0x00007f0d95e58620)), HDF5.Reference(r=HDF5.API.hobj_ref_t(buf=0x0000000000000001)), HDF5.Reference(r=HDF5.API.hobj_ref_t(buf=0x00007f0d95e58620)), HDF5.Reference(r=HDF5.API.hobj_ref_t(buf=0x0000000000000001)), HDF5.Reference(r=HDF5.API.hobj_ref_t(buf=0x00007f0d99cbc9d0)), HDF5.Reference(r=HDF5.API.hobj_ref_t(buf=0x0000000000000000))], vals=Array{WeakRef, (16,)}[#<null>, #<null>, #<null>, #<null>, #<null>, #<null>, #<null>, #<null>, #<null>, #<null>, #<null>, #<null>, #<null>, #<null>, #<null>, #<null>], ndel=0, count=0, age=0x0000000000000000, idxfloor=16, maxprobe=0), truncatemodules=Array{String, (0,)}[], gref=#<null>, nrefs=139696356454160), 0x00007f0d45773098), world=0x0000000000000001)
jl_method_error_bare at /cache/build/builder-amdci4-6/julialang/julia-release-1-dot-10/src/gf.c:2208
jl_method_error at /cache/build/builder-amdci4-6/julialang/julia-release-1-dot-10/src/gf.c:2226
jl_lookup_generic_ at /cache/build/builder-amdci4-6/julialang/julia-release-1-dot-10/src/gf.c:3057 [inlined]
ijl_apply_generic at /cache/build/builder-amdci4-6/julialang/julia-release-1-dot-10/src/gf.c:3072
read_vals_default at /home/erouviere/.julia/packages/JLD/GHujm/src/JLD.jl:478
read_vals at /home/erouviere/.julia/packages/JLD/GHujm/src/JLD.jl:455
unknown function (ip: 0x7f0d8b60d112)
_jl_invoke at /cache/build/builder-amdci4-6/julialang/julia-release-1-dot-10/src/gf.c:2894 [inlined]
ijl_apply_generic at /cache/build/builder-amdci4-6/julialang/julia-release-1-dot-10/src/gf.c:3076
read_array at /home/erouviere/.julia/packages/JLD/GHujm/src/JLD.jl:432
unknown function (ip: 0x7f0d8b5ffe5f)
_jl_invoke at /cache/build/builder-amdci4-6/julialang/julia-release-1-dot-10/src/gf.c:2894 [inlined]
ijl_apply_generic at /cache/build/builder-amdci4-6/julialang/julia-release-1-dot-10/src/gf.c:3076
read_array at /home/erouviere/.julia/packages/JLD/GHujm/src/JLD.jl:429
read at /home/erouviere/.julia/packages/JLD/GHujm/src/JLD.jl:388
read at /home/erouviere/.julia/packages/JLD/GHujm/src/JLD.jl:362
#48 at /home/erouviere/.julia/packages/JLD/GHujm/src/JLD.jl:1335 [inlined]
#jldopen#14 at /home/erouviere/.julia/packages/JLD/GHujm/src/JLD.jl:262
jldopen at /home/erouviere/.julia/packages/JLD/GHujm/src/JLD.jl:259 [inlined]
fileio_load at /home/erouviere/.julia/packages/JLD/GHujm/src/JLD.jl:1334
unknown function (ip: 0x7f0d8b5ff6e9)
_jl_invoke at /cache/build/builder-amdci4-6/julialang/julia-release-1-dot-10/src/gf.c:2894 [inlined]
ijl_apply_generic at /cache/build/builder-amdci4-6/julialang/julia-release-1-dot-10/src/gf.c:3076
jl_apply at /cache/build/builder-amdci4-6/julialang/julia-release-1-dot-10/src/julia.h:1982 [inlined]
jl_f__call_latest at /cache/build/builder-amdci4-6/julialang/julia-release-1-dot-10/src/builtins.c:812
#invokelatest#2 at ./essentials.jl:887 [inlined]
invokelatest at ./essentials.jl:884 [inlined]
#action#33 at /home/erouviere/.julia/packages/FileIO/jMf68/src/loadsave.jl:219
action at /home/erouviere/.julia/packages/FileIO/jMf68/src/loadsave.jl:196 [inlined]
#action#32 at /home/erouviere/.julia/packages/FileIO/jMf68/src/loadsave.jl:185 [inlined]
action at /home/erouviere/.julia/packages/FileIO/jMf68/src/loadsave.jl:185 [inlined]
#load#14 at /home/erouviere/.julia/packages/FileIO/jMf68/src/loadsave.jl:113
load at /home/erouviere/.julia/packages/FileIO/jMf68/src/loadsave.jl:109
unknown function (ip: 0x7f0d8b5fe0e9)
_jl_invoke at /cache/build/builder-amdci4-6/julialang/julia-release-1-dot-10/src/gf.c:2894 [inlined]
ijl_apply_generic at /cache/build/builder-amdci4-6/julialang/julia-release-1-dot-10/src/gf.c:3076
jl_apply at /cache/build/builder-amdci4-6/julialang/julia-release-1-dot-10/src/julia.h:1982 [inlined]
do_call at /cache/build/builder-amdci4-6/julialang/julia-release-1-dot-10/src/interpreter.c:126
eval_value at /cache/build/builder-amdci4-6/julialang/julia-release-1-dot-10/src/interpreter.c:223
eval_stmt_value at /cache/build/builder-amdci4-6/julialang/julia-release-1-dot-10/src/interpreter.c:174 [inlined]
eval_body at /cache/build/builder-amdci4-6/julialang/julia-release-1-dot-10/src/interpreter.c:617
jl_interpret_toplevel_thunk at /cache/build/builder-amdci4-6/julialang/julia-release-1-dot-10/src/interpreter.c:775
jl_toplevel_eval_flex at /cache/build/builder-amdci4-6/julialang/julia-release-1-dot-10/src/toplevel.c:934
jl_toplevel_eval_flex at /cache/build/builder-amdci4-6/julialang/julia-release-1-dot-10/src/toplevel.c:877
ijl_toplevel_eval_in at /cache/build/builder-amdci4-6/julialang/julia-release-1-dot-10/src/toplevel.c:985
eval at ./boot.jl:385 [inlined]
eval_user_input at /cache/build/builder-amdci4-6/julialang/julia-release-1-dot-10/usr/share/julia/stdlib/v1.10/REPL/src/REPL.jl:150
repl_backend_loop at /cache/build/builder-amdci4-6/julialang/julia-release-1-dot-10/usr/share/julia/stdlib/v1.10/REPL/src/REPL.jl:246
#start_repl_backend#46 at /cache/build/builder-amdci4-6/julialang/julia-release-1-dot-10/usr/share/julia/stdlib/v1.10/REPL/src/REPL.jl:231
start_repl_backend at /cache/build/builder-amdci4-6/julialang/julia-release-1-dot-10/usr/share/julia/stdlib/v1.10/REPL/src/REPL.jl:228
_jl_invoke at /cache/build/builder-amdci4-6/julialang/julia-release-1-dot-10/src/gf.c:2894 [inlined]
ijl_apply_generic at /cache/build/builder-amdci4-6/julialang/julia-release-1-dot-10/src/gf.c:3076
#run_repl#59 at /cache/build/builder-amdci4-6/julialang/julia-release-1-dot-10/usr/share/julia/stdlib/v1.10/REPL/src/REPL.jl:389
run_repl at /cache/build/builder-amdci4-6/julialang/julia-release-1-dot-10/usr/share/julia/stdlib/v1.10/REPL/src/REPL.jl:375
jfptr_run_repl_91689.1 at /home/erouviere/.julia/juliaup/julia-1.10.0+0.x64.linux.gnu/lib/julia/sys.so (unknown line)
_jl_invoke at /cache/build/builder-amdci4-6/julialang/julia-release-1-dot-10/src/gf.c:2894 [inlined]
ijl_apply_generic at /cache/build/builder-amdci4-6/julialang/julia-release-1-dot-10/src/gf.c:3076
#1013 at ./client.jl:432
jfptr_YY.1013_82677.1 at /home/erouviere/.julia/juliaup/julia-1.10.0+0.x64.linux.gnu/lib/julia/sys.so (unknown line)
_jl_invoke at /cache/build/builder-amdci4-6/julialang/julia-release-1-dot-10/src/gf.c:2894 [inlined]
ijl_apply_generic at /cache/build/builder-amdci4-6/julialang/julia-release-1-dot-10/src/gf.c:3076
jl_apply at /cache/build/builder-amdci4-6/julialang/julia-release-1-dot-10/src/julia.h:1982 [inlined]
jl_f__call_latest at /cache/build/builder-amdci4-6/julialang/julia-release-1-dot-10/src/builtins.c:812
#invokelatest#2 at ./essentials.jl:887 [inlined]
invokelatest at ./essentials.jl:884 [inlined]
run_main_repl at ./client.jl:416
exec_options at ./client.jl:333
_start at ./client.jl:552
jfptr__start_82703.1 at /home/erouviere/.julia/juliaup/julia-1.10.0+0.x64.linux.gnu/lib/julia/sys.so (unknown line)
_jl_invoke at /cache/build/builder-amdci4-6/julialang/julia-release-1-dot-10/src/gf.c:2894 [inlined]
ijl_apply_generic at /cache/build/builder-amdci4-6/julialang/julia-release-1-dot-10/src/gf.c:3076
jl_apply at /cache/build/builder-amdci4-6/julialang/julia-release-1-dot-10/src/julia.h:1982 [inlined]
true_main at /cache/build/builder-amdci4-6/julialang/julia-release-1-dot-10/src/jlapi.c:582
jl_repl_entrypoint at /cache/build/builder-amdci4-6/julialang/julia-release-1-dot-10/src/jlapi.c:731
main at /cache/build/builder-amdci4-6/julialang/julia-release-1-dot-10/cli/loader_exe.c:58
__libc_start_main at /lib/x86_64-linux-gnu/libc.so.6 (unknown line)