search

ggebbie / ECCOtour.jl

Take a self-guided tour of the ECCO version 4 release 4 ocean reanalysis product.
MIT License
1 stars 0 forks source link

NetCDF error when multithreading #71

Closed ggebbie closed 6 months ago

ggebbie commented 6 months ago 
[268256] signal (11.1): Segmentation fault
in expression starting at none:1
NC_create at /home/gebbie/.julia/artifacts/4a6fe8c6eda7f19d80afb858eb9f7cbe312f1453/lib/libnetcdf.so (unknown line)
nc__create at /home/gebbie/.julia/artifacts/4a6fe8c6eda7f19d80afb858eb9f7cbe312f1453/lib/libnetcdf.so (unknown line)
nc_create at /home/gebbie/.julia/artifacts/4a6fe8c6eda7f19d80afb858eb9f7cbe312f1453/lib/libnetcdf.so (unknown line)
nc_create at /home/gebbie/.julia/packages/NetCDF/7hOe9/src/netcdf_c.jl:34
nc_create at /home/gebbie/.julia/packages/NetCDF/7hOe9/src/netcdf_helpers.jl:110 [inlined]
#create#48 at /home/gebbie/.julia/packages/NetCDF/7hOe9/src/NetCDF.jl:772
create at /home/gebbie/.julia/packages/NetCDF/7hOe9/src/NetCDF.jl:763 [inlined]
#create#49 at /home/gebbie/.julia/packages/NetCDF/7hOe9/src/NetCDF.jl:854 [inlined]
create at /home/gebbie/.julia/packages/NetCDF/7hOe9/src/NetCDF.jl:854
unknown function (ip: 0x7f650e3ff660)
_jl_invoke at /cache/build/builder-amdci5-1/julialang/julia-release-1-dot-10/src/gf.c:2894 [inlined]
ijl_apply_generic at /cache/build/builder-amdci5-1/julialang/julia-release-1-dot-10/src/gf.c:3076
jl_apply at /cache/build/builder-amdci5-1/julialang/julia-release-1-dot-10/src/julia.h:1982 [inlined]
do_apply at /cache/build/builder-amdci5-1/julialang/julia-release-1-dot-10/src/builtins.c:768
#create#53 at /home/gebbie/.julia/packages/NetCDF/7hOe9/src/NetCDF.jl:1020
create at /home/gebbie/.julia/packages/NetCDF/7hOe9/src/NetCDF.jl:1019 [inlined]
#nccreate#71 at /home/gebbie/.julia/packages/NetCDF/7hOe9/src/NetCDF.jl:1268
_jl_invoke at /cache/build/builder-amdci5-1/julialang/julia-release-1-dot-10/src/gf.c:2894 [inlined]
ijl_apply_generic at /cache/build/builder-amdci5-1/julialang/julia-release-1-dot-10/src/gf.c:3076
jl_apply at /cache/build/builder-amdci5-1/julialang/julia-release-1-dot-10/src/julia.h:1982 [inlined]
do_apply at /cache/build/builder-amdci5-1/julialang/julia-release-1-dot-10/src/builtins.c:768
nccreate at /home/gebbie/.julia/packages/NetCDF/7hOe9/src/NetCDF.jl:1195
_jl_invoke at /cache/build/builder-amdci5-1/julialang/julia-release-1-dot-10/src/gf.c:2894 [inlined]
ijl_apply_generic at /cache/build/builder-amdci5-1/julialang/julia-release-1-dot-10/src/gf.c:3076
write at /home/gebbie/projects/ECCOtour.jl/src/ECCOtour.jl:1501
unknown function (ip: 0x7f650e3f7945)
_jl_invoke at /cache/build/builder-amdci5-1/julialang/julia-release-1-dot-10/src/gf.c:2894 [inlined]
ijl_apply_generic at /cache/build/builder-amdci5-1/julialang/julia-release-1-dot-10/src/gf.c:3076
macro expansion at ./timing.jl:279 [inlined]
macro expansion at /home/gebbie/projects/ECCOonPoseidon/scripts/state2sigmaregularpoles.jl:81 [inlined]
#215#threadsfor_fun#3 at ./threadingconstructs.jl:215
#215#threadsfor_fun at ./threadingconstructs.jl:182 [inlined]
#1 at ./threadingconstructs.jl:154
unknown function (ip: 0x7f650e381522)
_jl_invoke at /cache/build/builder-amdci5-1/julialang/julia-release-1-dot-10/src/gf.c:2894 [inlined]
ijl_apply_generic at /cache/build/builder-amdci5-1/julialang/julia-release-1-dot-10/src/gf.c:3076
jl_apply at /cache/build/builder-amdci5-1/julialang/julia-release-1-dot-10/src/julia.h:1982 [inlined]
start_task at /cache/build/builder-amdci5-1/julialang/julia-release-1-dot-10/src/task.c:1238
Allocations: 4874807464 (Pool: 4861515777; Big: 13291687); GC: 1638
ggebbie commented 6 months ago

when using ReentrantLock from alexander barth https://github.com/Alexander-Barth/NCDatasets.jl/issues/238

there is a python-related error

[271628] signal (11.1): Segmentation fault
in expression starting at REPL[21]:1
_PyInterpreterState_GET at /usr/local/src/conda/python-3.12.2/Include/internal/pycore_pystate.h:133 [inlined]
get_state at /usr/local/src/conda/python-3.12.2/Objects/obmalloc.c:866 [inlined]
_PyObject_Free at /usr/local/src/conda/python-3.12.2/Objects/obmalloc.c:1850 [inlined]
PyObject_Free at /usr/local/src/conda/python-3.12.2/Objects/obmalloc.c:830 [inlined]
unicode_dealloc at /usr/local/src/conda/python-3.12.2/Objects/unicodeobject.c:1612
datafile: state_3d_set1.0000105924.data  Thread ID: 4
year 2004 month 1
datafile: state_3d_set1.0000140988.data  Thread ID: 18
datafile: state_3d_set1.0000217692.data  Thread ID: 6
datafile: state_3d_set1.0000184836.data  Thread ID: 7
year 2016 month 10
year 2013 month 1
year 2008 month 1
Py_DecRef at /home/gebbie/.julia/packages/PythonCall/qTEA1/src/cpython/pointers.jl:299 [inlined]
#3 at /home/gebbie/.julia/packages/PythonCall/qTEA1/src/gc.jl:57 [inlined]
with_gil at /home/gebbie/.julia/packages/PythonCall/qTEA1/src/cpython/gil.jl:10 [inlined]
enqueue at /home/gebbie/.julia/packages/PythonCall/qTEA1/src/gc.jl:56
py_finalizer at /home/gebbie/.julia/packages/PythonCall/qTEA1/src/Py.jl:46
unknown function (ip: 0x7f797b48cb65)
datafile: state_3d_set1.0000162876.data  Thread ID: 15
datafile: state_3d_set1.0000151908.data  Thread ID: 17
year 2010 month 7
year 2009 month 4
_jl_invoke at /cache/build/builder-amdci5-1/julialang/julia-release-1-dot-10/src/gf.c:2894 [inlined]
ijl_apply_generic at /cache/build/builder-amdci5-1/julialang/julia-release-1-dot-10/src/gf.c:3076
run_finalizer at /cache/build/builder-amdci5-1/julialang/julia-release-1-dot-10/src/gc.c:318
datafile: state_3d_set1.0000047460.data  Thread ID: 14
datafile: state_3d_set1.0000094212.data  Thread ID: 5
datafile: state_3d_set1.0000024084.data  Thread ID: 16
year 1997 month 5
datafile: state_3d_set1.0000082524.data  Thread ID: 8
year 1994 month 9
year 2002 month 9
year 2001 month 5
jl_gc_run_finalizers_in_list at /cache/build/builder-amdci5-1/julialang/julia-release-1-dot-10/src/gc.c:408
run_finalizers at /cache/build/builder-amdci5-1/julialang/julia-release-1-dot-10/src/gc.c:454
jl_mutex_unlock at /cache/build/builder-amdci5-1/julialang/julia-release-1-dot-10/src/julia_locks.h:80 [inlinedjl_generate_fptr_impl at /cache/build/builder-amdci5-1/julialang/julia-release-1-dot-10/src/jitlayers.cpp:525
jl_compile_method_internal at /cache/build/builder-amdci5-1/julialang/julia-release-1-dot-10/src/gf.c:2480 [inlned]
jl_compile_method_internal at /cache/build/builder-amdci5-1/julialang/julia-release-1-dot-10/src/gf.c:2368
_jl_invoke at /cache/build/builder-amdci5-1/julialang/julia-release-1-dot-10/src/gf.c:2886 [inlined]
ijl_apply_generic at /cache/build/builder-amdci5-1/julialang/julia-release-1-dot-10/src/gf.c:3076
jl_apply at /cache/build/builder-amdci5-1/julialang/julia-release-1-dot-10/src/julia.h:1982 [inlined]
start_task at /cache/build/builder-amdci5-1/julialang/julia-release-1-dot-10/src/task.c:1238
Allocations: 55040724 (Pool: 54921114; Big: 119610); GC: 51
Segmentation fault (core dumped)
ggebbie commented 6 months ago

runs when PythonPlot, PythonCall removed

replace PythonPlot PythonCall with GeoPythonPlot and move to an extension