JuliaIO / JLD2.jl

HDF5-compatible file format in pure Julia
Other
550 stars 85 forks source link

EXCEPTION_ACCESS_VIOLATION with basic example #384

Closed gfrt0 closed 1 year ago

gfrt0 commented 2 years ago

Code:

using JLD2

x = 1
y = 2
z = [x,y,[x,y]]
str = "Hello World"
tup = (2,3,(4,5))
dict = Dict([("A", 1), ("B", 2)])

@save "variables.jld2"

<restart Julia>

using JLD2

@load "variables.jld2"

Error (\jldexample.jl:14 is the @load line; @save seems to work just fine):

Please submit a bug report with steps to reproduce this fault, and any error messages that follow (in their entirety). Thanks.
Exception: EXCEPTION_ACCESS_VIOLATION at 0xff4b01 -- gc_mark_loop at /cygdrive/c/buildbot/worker/package_win64/build/src\gc.c:2394 in expression starting at Julia\jldexample.jl:14 
gc_mark_loop at /cygdrive/c/buildbot/worker/package_win64/build/src\gc.c:2394
_jl_gc_collect at /cygdrive/c/buildbot/worker/package_win64/build/src\gc.c:3039
jl_gc_collect at /cygdrive/c/buildbot/worker/package_win64/build/src\gc.c:3248
maybe_collect at /cygdrive/c/buildbot/worker/package_win64/build/src\gc.c:882 [inlined]
jl_gc_pool_alloc at /cygdrive/c/buildbot/worker/package_win64/build/src\gc.c:1209
jl_gc_alloc_ at /cygdrive/c/buildbot/worker/package_win64/build/src\julia_internal.h:339 [inlined]
jl_gc_alloc at /cygdrive/c/buildbot/worker/package_win64/build/src\gc.c:3290
_new_array_ at /cygdrive/c/buildbot/worker/package_win64/build/src\array.c:127 [inlined]
_new_array at /cygdrive/c/buildbot/worker/package_win64/build/src\array.c:193 [inlined]
jl_alloc_array_1d at /cygdrive/c/buildbot/worker/package_win64/build/src\array.c:441
Array at .\boot.jl:457 [inlined]
Array at .\boot.jl:466 [inlined]
similar at .\abstractarray.jl:829 [inlined]
similar at .\abstractarray.jl:828 [inlined]
_array_for at .\array.jl:613 [inlined]
_array_for at .\array.jl:616 [inlined]
IncrementalCompact at .\compiler/ssair\ir.jl:575
IncrementalCompact at .\compiler/ssair\ir.jl:575 [inlined]
getfield_elim_pass! at .\compiler/ssair\passes.jl:526
run_passes at .\compiler\optimize.jl:311
optimize at .\compiler\optimize.jl:296 [inlined]
_typeinf at .\compiler\typeinfer.jl:255
typeinf at .\compiler\typeinfer.jl:209
abstract_call_method_with_const_args at .\compiler\abstractinterpretation.jl:557
abstract_call_gf_by_type at .\compiler\abstractinterpretation.jl:113
abstract_call_known at .\compiler\abstractinterpretation.jl:1342
abstract_call at .\compiler\abstractinterpretation.jl:1397
abstract_call at .\compiler\abstractinterpretation.jl:1382
abstract_eval_statement at .\compiler\abstractinterpretation.jl:1534
typeinf_local at .\compiler\abstractinterpretation.jl:1918
typeinf_nocycle at .\compiler\abstractinterpretation.jl:2014
_typeinf at .\compiler\typeinfer.jl:226
typeinf at .\compiler\typeinfer.jl:209
abstract_call_method_with_const_args at .\compiler\abstractinterpretation.jl:557
abstract_call_gf_by_type at .\compiler\abstractinterpretation.jl:113
abstract_call_known at .\compiler\abstractinterpretation.jl:1342
abstract_call at .\compiler\abstractinterpretation.jl:1397
abstract_call at .\compiler\abstractinterpretation.jl:1382
abstract_eval_statement at .\compiler\abstractinterpretation.jl:1534
typeinf_local at .\compiler\abstractinterpretation.jl:1918
typeinf_nocycle at .\compiler\abstractinterpretation.jl:2014
_typeinf at .\compiler\typeinfer.jl:226
typeinf at .\compiler\typeinfer.jl:209
typeinf_ext at .\compiler\typeinfer.jl:909
typeinf_ext_toplevel at .\compiler\typeinfer.jl:942
typeinf_ext_toplevel at .\compiler\typeinfer.jl:938
jfptr_typeinf_ext_toplevel_16182.clone_1 at C:\Users\gfrt0\AppData\Local\Programs\Julia-1.7.2\lib\julia\sys.dll (unknown line)
_jl_invoke at /cygdrive/c/buildbot/worker/package_win64/build/src\gf.c:2247 [inlined]
jl_apply_generic at /cygdrive/c/buildbot/worker/package_win64/build/src\gf.c:2429 [inlined]
jl_apply at /cygdrive/c/buildbot/worker/package_win64/build/src\julia.h:1788 [inlined]
jl_type_infer at /cygdrive/c/buildbot/worker/package_win64/build/src\gf.c:295
jl_generate_fptr at /cygdrive/c/buildbot/worker/package_win64/build/src\jitlayers.cpp:338
jl_compile_method_internal at /cygdrive/c/buildbot/worker/package_win64/build/src\gf.c:1980
jl_compile_method_internal at /cygdrive/c/buildbot/worker/package_win64/build/src\gf.c:1934 [inlined]
_jl_invoke at /cygdrive/c/buildbot/worker/package_win64/build/src\gf.c:2239 [inlined]
jl_apply_generic at /cygdrive/c/buildbot/worker/package_win64/build/src\gf.c:2429
read_data at C:\Users\gfrt0\.julia\packages\JLD2\MXv8x\src\datasets.jl:170
unknown function (ip: 000000005fc65b5c)
read_data at C:\Users\gfrt0\.julia\packages\JLD2\MXv8x\src\datasets.jl:149
unknown function (ip: 000000005fc5a375)
unknown function (ip: 000000005fc58d89)
Allocations: 19251923 (Pool: 19242910; Big: 9013); GC: 22

Reinstalling JLD2 with Pkg.build("JLD2") and retrying yields a similar error, but a different unknown function:

jlconvert at C:\Users\gfrt0\.julia\packages\JLD2\MXv8x\src\data\reconstructing_datatypes.jl:326  
unknown function (ip: 000000005fd3ba2c)  
unknown function (ip: 000000005fd3b159)  

Apologies if this is a Julia problem and not a JLD2 problem. Given reinstalling returns a different error I suspect this may be a problem with my build.

JonasIsensee commented 2 years ago

Hi @gfrt0 ,

I can reproduce this issue with julia 1.7.2 and the latest release of JLD2. The issue is this (##meta#62 doc string container object, that can't be properly serialized / deserialized)

julia> f = jldopen("variables.jld2")
JLDFile /mnt/c/Users/Jonas Isensee/variables.jld2 (read-only)
 ├─🔢 ##meta#62
 ├─🔢 dict
 ├─🔢 str
 ├─🔢 tup
 ├─🔢 x
 ├─🔢 y
 └─🔢 z

@save without variables as arguments stores everything it finds in the main scope. This brings all kinds of problems with it which is also the reason that I've removed @save and @load from the docs. (e.g. silently deprecated so far)

I already tried to fix this once at some point in the past but the variable name seems to have changed: https://github.com/JuliaIO/JLD2.jl/blob/abb9e5920bbe956a4d9fd2f92550cd7ea0a715aa/src/loadsave.jl#L85

If you want, you could open a PR to make this line match more broadly.

gfrt0 commented 2 years ago

Thank you @JonasIsensee, I noticed I could not find @save without arguments in the docs but did not think much of it. I can see how it can go wrong. I will find another way to list the large number of objects I want to save.