search

JuliaIO / JLD2.jl

HDF5-compatible file format in pure Julia
Other
560 stars 92 forks source link

Access violation error message beginning "Please submit a bug report with steps to reproduce this fault..." #312

Closed JRStevensHHS closed 3 years ago

JRStevensHHS commented 3 years ago

Hello, I have recieved an error message on my code as follows:

Please submit a bug report with steps to reproduce this fault, and any error messages that follow (in their entirety). Thanks.
Exception: EXCEPTION_ACCESS_VIOLATION at 0x5c20c405 --  at 0x5c20c405 -- g report with steps to reproduce this fault, and any error messages that follow (in their entirety). Thanks.
Exception: EXCEPTION_ACCESS_VIOLATION at 0x5c20c405 --  at 0x5c20c405 -- OLATION:105 [inlined]
unsafe_load at .\pointer.jl:105 [inlined]
unsafe_load at .\pointer.jl:105 [inlined]
jlunsafe_load at C:\Users\JacobStevens\.julia\packages\JLD2\KN6F6\src\JLD2.jl:43 [inlined]
jlunsafe_load at C:\Users\JacobStevens\.julia\packages\JLD2\KN6F6\src\misc.jl:19 [inlined]
_read at C:\Users\JacobStevens\.julia\packages\JLD2\KN6F6\src\mmapio.jl:215 [inlined]
jlread at C:\Users\JacobStevens\.julia\packages\JLD2\KN6F6\src\misc.jl:28 [inlined]
read_obj_start at C:\Users\JacobStevens\.julia\packages\JLD2\KN6F6\src\object_headers.jl:46
in expression starting at C:\Users\JacobStevens\OneDrive - Handelsh├Âgskolan i Stockholm\Thesis_Julia - Dynamic Labour\obtain_exo.jl:70
unsafe_load at .\pointer.jl:105 [inlined]
jlunsafe_load at C:\Users\JacobStevens\.julia\packages\JLD2\KN6F6\src\JLD2.jl:43 [inlined]
jlunsafe_load at C:\Users\JacobStevens\.julia\packages\JLD2\KN6F6\src\misc.jl:19 [inlined]
_read at C:\Users\JacobStevens\.julia\packages\JLD2\KN6F6\src\mmapio.jl:215 [inlined]
jlread at C:\Users\JacobStevens\.julia\pacunsafe_load at .\pointer.jl:105 [inlined]
jlunsafe_load at C:\Users\JacobStevens\.julia\packages\JLD2\KN6F6\src\JLD2.jl:43 [inlined]
jlunsafe_load at C:\Users\JacobStevens\.julia\packages\JLD2\KN6F6\src\misc.jl:19 [inread_obj_start at C:\Users\JacobStevens\.julia\packages\JLD2\KN6F6\src\object_headers.jl:46
in expression starting at C:\Users\JacobStevens\OneDrive - Handelsh├Âgskolan i Stockholm\Th_read at C:\Users\JacobStevens\.julia\packages\JLD2\KN6F6\src\mmapio.jl:215 [inlined]
jlread at C:\Users\JacobStevens\.julia\packages\JLD2\KN6F6\src\misc.jl:28 [inlined]
read_obj_start at C:\Users\JacobStevens\.julia\packages\JLD2\KN6F6\src\object_headers.jl:46
read_obj_start at C:\Users\JacobStevens\.julia\packages\JLD2\KN6F6\src\object_headers.jl:46
is_Julia - Dynamic Labour\obtain_exo.jl:70
isgroup at C:\Users\JacobStevens\.julia\packages\JLD2\KN6F6\src\object_headers.jl:74
unsafe_load at .\pointer.jl:105 [inlined]
unsafe_load at .\pointer.jl:105 [inlined]
jlunsafe_load at C:\Users\JacobStevens\.julia\packages\JLD2\KN6F6\src\JLD2.jl:43 [inlined]
jlunsafe_load at C:\Users\JacobStevens\.julia\packages\JLD2\KN6F6\src\misc.jl:19 [inlined]
_read at C:\Users\JacobStevens\.julia\packages\JLD2\KN6F6\src\mmapio.jl:215 [inlined]
jlread at C:\Users\JacobStevens\.julia\packages\JLD2\KN6F6\src\misc.jl:28 [inlined]
read_obj_start at C:\Users\JacobStevens\.julia\packages\JLD2\KN6F6\src\object_headers.jl:46
read_obj_start at C:\Users\JacobStevens\.julia\packages\JLD2\KN6F6\src\object_hisgroup at C:\Users\JacobStevens\.julia\packages\JLD2\KN6F6\src\object_headers.jl:74
getindex at C:\Users\JacobStevens\.julia\packages\JLD2\KN6F6\src\groups.jl:103
read at C:\Users\JacobStevens\.julia\packages\JLD2\KN6F6\src\JLD2.jl:374
#65 at C:\Users\JacobStevens\.julia\packages\JLD2\KN6F6\src\fileio.jl:44
#65 at C:\Users\JacobStevens\.julia\packages\JLD2\KN6F6\src\fileio.jl:44
unknown function (ip: 000000005c211036)
unknown function (ip: 000000005c211036)
ges\JLD2\KN6F6\src\fileio.jl:44
unknown function (ip: 000000005c211036)
#jldopen#40 at C:\Users\JacobStevens\.julia\packages\JLD2\KN6F6\src\loadsave.jl:4
#jldopen#40 at C:\Users\JacobStevens\.julia\packages\JLD2\KN6F6\src\loadsave.jl:4
ldopen at C:\Users\JacobStevens\.julia\packages\JLD2\KN6F6\src\loadsave.jl:2 [inlined]
#fileio_load#64 at C:\Users\JacobStevens\.julia\packages\JLD2\KN6F6\src\fileio.jl:43 [inlined]
fileio_load at C:\Users\JacobStevens\.julia\packages\JLD2\KN6F6\src\fileio.jl:43
unknown function (ip: 000000005c202cf4)
unknown function (ip: 000000005c202cf4)
ia\packages\JLD2\KN6F6\src\fileio.jl:43
43 [in#fileio_load#64 at C:\Users\JacobStevens\.julia\packages\JLD2\KN6F6\src\fileio.jl:43 [inlined]
fileio_load at C:\Users\JacobStevens\.julia\packages\JLD2\KN6F6\src\fileio.jl:43
unknown function (ip: 000000005c202cf4)
jl_f__call_latest at C:\Users\JacobStevens\AppData\Local\Programs\Julia-1.6.0\bin\libjulia-internal.dll (unknown line)
jl_apply at /cygdrive/c/buildbot/worker/package_win64/build/src\julia.h:1703 [inlined]
jl_f__call_latest at /cygdrive/c/buildbot/worker/package_win64/build/src\builtins.c:714
#invokelatest#2 at .\essentials.jl:708 [inlined]
invokelatest at .\essentials.jl:706 [inlined]
#action#33 at C:\Users\JacobStevens\.julia\packages\FileIO\0RlLC\src\loadsave.jl:212
#action#33 at C:\Users\JacobStevens\.julia\packaginvokelatest at .\essentials.jl:706 [inlined]
\FileIO\0RlLC\src\loadsave.jl:192 [inlined]
#action#32 at C:\Users\JacobStevens\.julia\pac#action#33 at C:\Users\JacobStevens\.julia\packages\FileIO\0RlLC\src\loadsave.jl:212
#action#33 at C:\Users\JacobStevens\.julia\packages\FileIO\0RlLC\src\loadsave.jl:212
action at C:\Users\JacobStevens\.julia\packages\FileIO\0RlLC\src\loadsave.jl:192 [inlined]
#action#32 at C:\Users\JacobStevens\.julia\packages\FileIO\0RlLC\src\loadsave.jl:181
action at C:\Users\JacobStevens\.julia\packages\FileIO\0RlLC\src\loadsave.jl:181 [inlined]
#load#14 at C:\Users\JacobStevens\.julia\packages\FileIO\0RlLC\src\loadsave.jl:113
#load#14 at C:\Users\JacobStevens\.julia\packages\FileIO\0RlLC\src\loadsave.jl:113
lined]
#load#14 at C:\Users\JacobStevens\.julia\packages\FileIO\0RlLC\src\loadsave.jl:113
load at C:\Users\JacobStevens\.julia\packages\FileIO\0RlLC\src\loadsave.jl:110 [inlined]
S20 at C:\Users\JacobStevens\OneDrive - Handelsh├Âgskolan i Stockholm\Thesis_Julia - Dynamic Labour\Solve_From_1981.jl:71
unknown function (ip: 00000000609d9539)
unknown function (ip: 00000000609d9539)
andelsh├Âgskolan i Stockholm\Thesis_Julia - DynaS20 at C:\Users\JacobStevens\OneDrive - Handelsh├Âgskolan i Stockholm\Thesis_Julia - Dynamic Labour\Solve_From_1981.jl:71
unknown function (ip: 00000000609d9539)
unknown function (ip: 00000000609d9539)
andelsh├Âgskolan i Stockholm\Thesis_Julia - Dynamic Labour\Solve_From_1981.jl:71
jl:73 [inlined]
#918#threadsfor_fun at .\threadingconstructs.jl:81
#918#threadsfor_fun at .\threadingconstructs.jl:81
Handelsh├Âgskolan i Stockholm\Thesis_Julia - Dynamic Labour\obtain_exo.jl:73 [inlined]
#918#threadsfor_fun at .\threadingconstructs.jl:81
#918#threadsfor_fun at .\threadingconstructs.jl:48
unknown function (ip: 00000000609d8643)
unknown function (ip: 00000000609d8643)
cts.jl:48
unknown function (ip: 00000000609d8643)
jl_apply at /cygdrive/c/buildbot/worker/package_win64/build/src\julia.h:1703 [inlined]
start_task at /cygdrive/c/buildbot/worker/package_win64/build/src\task.c:839
Allocations: 317946924 (Pool: 313606241; Big: 4340683); GC: 388
Allocations: 317946924 (Pool: 313606241; Big: 4340683); GC: 388
\task.c:839
inlined]
start_task at /cygdrive/c/buildbot/worker/package_win64/build/src\task.c:839
Allocations: 317946924 (Pool: 313606241; Big: 4340683); GC: 388

The relevant code runs:

for z ∈ 1:sktp
    Threads.@threads for x ∈ 16:41
        local s20 = pop_path_nobabies(x,x,z,w, κ); # of course, y = x for this purpose
        R20 =  S20(s20, AT, res, runs - 38)["debt"]

        # extract debt info
        global Ω[x,z,w] = R20[end];
        save(string("Omega", w, ".jld2"), "Omega", Ω)
        println("finished ", z, ", ", x)
    end
end

Where "S20(...)" is a long, complicated function that begins by loading a jld2 file as the initial guess for a nonlinear equation solver.

System specs:

My guess is that this is being caused by the threading - the parallel cores try to load the same .jld2 at the same time, and this caused the access violation. But this is only a guess; I have been running versions of this code for months and this is the first time the error has arisen. Indeed the error only arises on my Windows 10 laptop, my Ubuntu server is running the code just fine as I type this. (same version of Julia).

JonasIsensee commented 3 years ago

Hi @JRStevensHHS , multi-threaded reading of the same file is not supported. I would recommend that you load the data in the file just once and pass the data to your threads.