JuliaLang / julia

The Julia Programming Language
https://julialang.org/
MIT License
45.43k stars 5.45k forks source link

How to drill into this crash #55696

Closed AUK1939 closed 1 week ago

AUK1939 commented 1 week ago

Hi I have a program that takes a long time to run, around 6 hours. It's just a loop that runs for a while. It used to work fine but now crashes (I'm not really sure what I've updated to cause this). I was hoping to get some help in tracking this down.

Everytime the program seems to crash at different locations - which is very odd because it used to work just fine. I've pasted some of the errors below.

Sometimes I get EXCEPTION_ACCESS_VIOLATIONS, sometimes I get OutOfMemory exceptions (which is really weird since I have 80 gbs free memory in task manager). Is there any way to drill down and figure out what's causing this? It would be good to produce a MWE but right now I'm not even sure how to achieve that given the program seems to be crashing in different places.

Please submit a bug report with steps to reproduce this fault, and any error messages that follow (in their entirety). Thanks.
Exception: EXCEPTION_ACCESS_VIOLATION at 0x7ffec1da8035 -- gc_try_claim_and_push at C:/workdir/src\gc.c:1841 [inlined]
gc_mark_objarray at C:/workdir/src\gc.c:1994
in expression starting at REPL[6]:1
gc_try_claim_and_push at C:/workdir/src\gc.c:1841 [inlined]
gc_mark_objarray at C:/workdir/src\gc.c:1994
gc_mark_outrefs at C:/workdir/src\gc.c:2602 [inlined]
gc_mark_loop_serial_ at C:/workdir/src\gc.c:2697
gc_mark_loop_serial at C:/workdir/src\gc.c:2720
gc_mark_loop at C:/workdir/src\gc.c:2901 [inlined]
_jl_gc_collect at C:/workdir/src\gc.c:3234
ijl_gc_collect at C:/workdir/src\gc.c:3531
maybe_collect at C:/workdir/src\gc.c:937 [inlined]
jl_gc_pool_alloc_inner at C:/workdir/src\gc.c:1300
jl_gc_pool_alloc_noinline at C:/workdir/src\gc.c:1357 [inlined]
jl_gc_alloc_ at C:/workdir/src\julia_internal.h:478 [inlined]
jl_gc_alloc at C:/workdir/src\gc.c:3583
_new_array_ at C:/workdir/src\array.c:134
_new_array at C:/workdir/src\array.c:198 [inlined]
ijl_alloc_array_1d at C:/workdir/src\array.c:436
Array at .\boot.jl:477 [inlined]
Array at .\boot.jl:486 [inlined]
zeros at .\array.jl:636 [inlined]
zeros at .\array.jl:632 [inlined]
rehash! at C:\Users\userx\.julia\packages\OrderedCollections\9C4Uz\src\ordered_dict.jl:138
_setindex! at C:\Users\userx\.julia\packages\OrderedCollections\9C4Uz\src\ordered_dict.jl:321
setindex! at C:\Users\userx\.julia\packages\OrderedCollections\9C4Uz\src\ordered_dict.jl:338 [inlined]
_add_or_set! at C:\Users\userx\.julia\packages\JuMP\PKxEW\src\aff_expr.jl:26
add_to_expression! at C:\Users\userx\.julia\packages\JuMP\PKxEW\src\aff_expr.jl:557 [inlined]
operate! at C:\Users\userx\.julia\packages\JuMP\PKxEW\src\mutable_arithmetics.jl:225 [inlined]
buffered_operate_fallback! at C:\Users\userx\.julia\packages\MutableArithmetics\OfRUD\src\interface.jl:558 [inlined]
buffered_operate! at C:\Users\userx\.julia\packages\MutableArithmetics\OfRUD\src\interface.jl:575 [inlined]
buffered_operate_fallback!! at C:\Users\userx\.julia\packages\MutableArithmetics\OfRUD\src\interface.jl:708 [inlined]
buffered_operate!! at C:\Users\userx\.julia\packages\MutableArithmetics\OfRUD\src\interface.jl:685 [inlined]
buffered_operate! at C:\Users\userx\.julia\packages\MutableArithmetics\OfRUD\src\implementations\LinearAlgebra.jl:330
operate! at C:\Users\userx\.julia\packages\MutableArithmetics\OfRUD\src\implementations\LinearAlgebra.jl:376 [inlined]
operate_to! at C:\Users\userx\.julia\packages\MutableArithmetics\OfRUD\src\implementations\LinearAlgebra.jl:394 [inlined]
_mul! at C:\Users\userx\.julia\packages\MutableArithmetics\OfRUD\src\dispatch.jl:130 [inlined]
mul! at C:\Users\userx\.julia\packages\MutableArithmetics\OfRUD\src\dispatch.jl:161 [inlined]
* at C:\workdir\usr\share\julia\stdlib\v1.10\LinearAlgebra\src\matmul.jl:57
* at C:\workdir\usr\share\julia\stdlib\v1.10\LinearAlgebra\src\matmul.jl:87 [inlined]
macro expansion at C:\Users\userx\.julia\packages\MutableArithmetics\OfRUD\src\rewrite.jl:340 [inlined]
macro expansion at C:\Users\userx\.julia\packages\JuMP\PKxEW\src\macros.jl:257 [inlined]
macro expansion at C:\Users\userx\.julia\packages\JuMP\PKxEW\src\macros\@constraint.jl:171 [inlined]
macro expansion at C:\Users\userx\.julia\packages\JuMP\PKxEW\src\macros.jl:393 [inlined]
optimizecrrautility at C:\dev\Apollo\src\volatility_premiums\option_portfolios\vrp_backtest.jl:248
positiveskewedportfolio at C:\dev\Apollo\src\volatility_premiums\option_portfolios\vrp_backtest.jl:421
optimalportfolio at C:\dev\Apollo\src\volatility_premiums\option_portfolios\vrp_backtest.jl:453
unknown function (ip: 0000028f165b3dd3)
volatilitypremiumsbacktest at C:\dev\Apollo\src\volatility_premiums\option_portfolios\vrp_backtest.jl:627
unknown function (ip: 0000028f11588e81)
jl_apply at C:/workdir/src\julia.h:1982 [inlined]
do_call at C:/workdir/src\interpreter.c:126
eval_value at C:/workdir/src\interpreter.c:223
eval_stmt_value at C:/workdir/src\interpreter.c:174 [inlined]
eval_body at C:/workdir/src\interpreter.c:635
jl_interpret_toplevel_thunk at C:/workdir/src\interpreter.c:775
jl_toplevel_eval_flex at C:/workdir/src\toplevel.c:934
jl_toplevel_eval_flex at C:/workdir/src\toplevel.c:877
jl_toplevel_eval_flex at C:/workdir/src\toplevel.c:877
eval_body at C:/workdir/src\interpreter.c:579
eval_body at C:/workdir/src\interpreter.c:544
jl_interpret_toplevel_thunk at C:/workdir/src\interpreter.c:775
jl_toplevel_eval_flex at C:/workdir/src\toplevel.c:934
ijl_toplevel_eval at C:/workdir/src\toplevel.c:943 [inlined]
ijl_toplevel_eval_in at C:/workdir/src\toplevel.c:985
eval at .\boot.jl:385 [inlined]
eval_user_input at C:\workdir\usr\share\julia\stdlib\v1.10\REPL\src\REPL.jl:150
repl_backend_loop at C:\workdir\usr\share\julia\stdlib\v1.10\REPL\src\REPL.jl:246
#start_repl_backend#46 at C:\workdir\usr\share\julia\stdlib\v1.10\REPL\src\REPL.jl:231
start_repl_backend at C:\workdir\usr\share\julia\stdlib\v1.10\REPL\src\REPL.jl:228
#run_repl#59 at C:\workdir\usr\share\julia\stdlib\v1.10\REPL\src\REPL.jl:389
run_repl at C:\workdir\usr\share\julia\stdlib\v1.10\REPL\src\REPL.jl:375
jfptr_run_repl_95862.1 at C:\Julia\Julia-1.10.5\lib\julia\sys.dll (unknown line)
#1013 at .\client.jl:432
jfptr_YY.1013_86638.1 at C:\Julia\Julia-1.10.5\lib\julia\sys.dll (unknown line)
jl_apply at C:/workdir/src\julia.h:1982 [inlined]
jl_f__call_latest at C:/workdir/src\builtins.c:812
#invokelatest#2 at .\essentials.jl:892 [inlined]
invokelatest at .\essentials.jl:889 [inlined]
run_main_repl at .\client.jl:416
exec_options at .\client.jl:333
_start at .\client.jl:552
jfptr__start_86663.1 at C:\Julia\Julia-1.10.5\lib\julia\sys.dll (unknown line)
jl_apply at C:/workdir/src\julia.h:1982 [inlined]
true_main at C:/workdir/src\jlapi.c:582
jl_repl_entrypoint at C:/workdir/src\jlapi.c:731
mainCRTStartup at C:/workdir/cli\loader_exe.c:58
BaseThreadInitThunk at C:\Windows\System32\KERNEL32.DLL (unknown line)
RtlUserThreadStart at C:\Windows\SYSTEM32\ntdll.dll (unknown line)
Allocations: 371361442 (Pool: 363078603; Big: 8282839); GC: 342
Please submit a bug report with steps to reproduce this fault, and any error messages that follow (in their entirety). Thanks.
Exception: EXCEPTION_ACCESS_VIOLATION at 0x7ffec2e7e86c -- R_ParseEvalString at C:\Program Files\R\R-4.3.3\bin\x64\R.dll (unknown line)
in expression starting at REPL[6]:1
R_ParseEvalString at C:\Program Files\R\R-4.3.3\bin\x64\R.dll (unknown line)
R_ParseEvalString at C:\Program Files\R\R-4.3.3\bin\x64\R.dll (unknown line)
R_getBCInterpreterExpression at C:\Program Files\R\R-4.3.3\bin\x64\R.dll (unknown line)
Rf_applyClosure at C:\Program Files\R\R-4.3.3\bin\x64\R.dll (unknown line)
R_ParseEvalString at C:\Program Files\R\R-4.3.3\bin\x64\R.dll (unknown line)
R_ParseEvalString at C:\Program Files\R\R-4.3.3\bin\x64\R.dll (unknown line)
R_getBCInterpreterExpression at C:\Program Files\R\R-4.3.3\bin\x64\R.dll (unknown line)
Rf_applyClosure at C:\Program Files\R\R-4.3.3\bin\x64\R.dll (unknown line)
R_ParseEvalString at C:\Program Files\R\R-4.3.3\bin\x64\R.dll (unknown line)
R_ParseEvalString at C:\Program Files\R\R-4.3.3\bin\x64\R.dll (unknown line)
R_getBCInterpreterExpression at C:\Program Files\R\R-4.3.3\bin\x64\R.dll (unknown line)
Rf_applyClosure at C:\Program Files\R\R-4.3.3\bin\x64\R.dll (unknown line)
R_ParseEvalString at C:\Program Files\R\R-4.3.3\bin\x64\R.dll (unknown line)
R_ParseEvalString at C:\Program Files\R\R-4.3.3\bin\x64\R.dll (unknown line)
R_getBCInterpreterExpression at C:\Program Files\R\R-4.3.3\bin\x64\R.dll (unknown line)
R_execMethod at C:\Program Files\R\R-4.3.3\bin\x64\R.dll (unknown line)
R_dispatchGeneric at C:\Program Files\R\R-4.3.3\library\methods\libs\x64\methods.dll (unknown line)
R_set_standardGeneric_ptr at C:\Program Files\R\R-4.3.3\bin\x64\R.dll (unknown line)
R_ParseEvalString at C:\Program Files\R\R-4.3.3\bin\x64\R.dll (unknown line)
R_ParseEvalString at C:\Program Files\R\R-4.3.3\bin\x64\R.dll (unknown line)
R_getBCInterpreterExpression at C:\Program Files\R\R-4.3.3\bin\x64\R.dll (unknown line)
Rf_applyClosure at C:\Program Files\R\R-4.3.3\bin\x64\R.dll (unknown line)
R_ParseEvalString at C:\Program Files\R\R-4.3.3\bin\x64\R.dll (unknown line)
R_new_custom_connection at C:\Program Files\R\R-4.3.3\bin\x64\R.dll (unknown line)
R_ToplevelExec at C:\Program Files\R\R-4.3.3\bin\x64\R.dll (unknown line)
R_tryEval at C:\Program Files\R\R-4.3.3\bin\x64\R.dll (unknown line)
#28 at C:\Users\userx\.julia\packages\RCall\FEbLj\src\eval.jl:88
disable_sigint at .\c.jl:473 [inlined]
tryEval at C:\Users\userx\.julia\packages\RCall\FEbLj\src\eval.jl:85 [inlined]
reval_p at C:\Users\userx\.julia\packages\RCall\FEbLj\src\eval.jl:100
unknown function (ip: 0000014758266b15)
reval_p at C:\Users\userx\.julia\packages\RCall\FEbLj\src\eval.jl:119
reval at C:\Users\userx\.julia\packages\RCall\FEbLj\src\eval.jl:136
macro expansion at C:\Users\userx\.julia\packages\RCall\FEbLj\src\macros.jl:67 [inlined]
armagarchscenerios at C:\dev\Apollo\src\volatility_premiums\option_portfolios\fhs_densities.jl:40
unknown function (ip: 000001479793b893)
generatepaths at C:\dev\Apollo\src\volatility_premiums\option_portfolios\op_utils.jl:6
unknown function (ip: 000001479792b4a0)
volatilitypremiumsbacktest at C:\dev\Apollo\src\volatility_premiums\option_portfolios\vrp_backtest.jl:609
unknown function (ip: 000001479c2e8e41)
jl_apply at C:/workdir/src\julia.h:1982 [inlined]
do_call at C:/workdir/src\interpreter.c:126
eval_value at C:/workdir/src\interpreter.c:223
eval_stmt_value at C:/workdir/src\interpreter.c:174 [inlined]
eval_body at C:/workdir/src\interpreter.c:635
jl_interpret_toplevel_thunk at C:/workdir/src\interpreter.c:775
jl_toplevel_eval_flex at C:/workdir/src\toplevel.c:934
jl_toplevel_eval_flex at C:/workdir/src\toplevel.c:877
jl_toplevel_eval_flex at C:/workdir/src\toplevel.c:877
eval_body at C:/workdir/src\interpreter.c:579
eval_body at C:/workdir/src\interpreter.c:544
jl_interpret_toplevel_thunk at C:/workdir/src\interpreter.c:775
jl_toplevel_eval_flex at C:/workdir/src\toplevel.c:934
ijl_toplevel_eval at C:/workdir/src\toplevel.c:943 [inlined]
ijl_toplevel_eval_in at C:/workdir/src\toplevel.c:985
eval at .\boot.jl:385 [inlined]
eval_user_input at C:\workdir\usr\share\julia\stdlib\v1.10\REPL\src\REPL.jl:150
repl_backend_loop at C:\workdir\usr\share\julia\stdlib\v1.10\REPL\src\REPL.jl:246
#start_repl_backend#46 at C:\workdir\usr\share\julia\stdlib\v1.10\REPL\src\REPL.jl:231
start_repl_backend at C:\workdir\usr\share\julia\stdlib\v1.10\REPL\src\REPL.jl:228
#run_repl#59 at C:\workdir\usr\share\julia\stdlib\v1.10\REPL\src\REPL.jl:389
run_repl at C:\workdir\usr\share\julia\stdlib\v1.10\REPL\src\REPL.jl:375
jfptr_run_repl_95862.1 at C:\Julia\Julia-1.10.5\lib\julia\sys.dll (unknown line)
#1013 at .\client.jl:432
jfptr_YY.1013_86638.1 at C:\Julia\Julia-1.10.5\lib\julia\sys.dll (unknown line)
jl_apply at C:/workdir/src\julia.h:1982 [inlined]
jl_f__call_latest at C:/workdir/src\builtins.c:812
#invokelatest#2 at .\essentials.jl:892 [inlined]
invokelatest at .\essentials.jl:889 [inlined]
run_main_repl at .\client.jl:416
exec_options at .\client.jl:333
_start at .\client.jl:552
jfptr__start_86663.1 at C:\Julia\Julia-1.10.5\lib\julia\sys.dll (unknown line)
jl_apply at C:/workdir/src\julia.h:1982 [inlined]
true_main at C:/workdir/src\jlapi.c:582
jl_repl_entrypoint at C:/workdir/src\jlapi.c:731
mainCRTStartup at C:/workdir/cli\loader_exe.c:58
BaseThreadInitThunk at C:\Windows\System32\KERNEL32.DLL (unknown line)
RtlUserThreadStart at C:\Windows\SYSTEM32\ntdll.dll (unknown line)
Allocations: 648598899 (Pool: 633463886; Big: 15135013); GC: 554
Please submit a bug report with steps to reproduce this fault, and any error messages that follow (in their entirety). Thanks.
Exception: EXCEPTION_ACCESS_VIOLATION at 0x14d299e6453 -- getproperty at .\Base.jl:37 [inlined]
iswritable at .\iostream.jl:73 [inlined]
write at .\iostream.jl:368
in expression starting at REPL[7]:1
getproperty at .\Base.jl:37 [inlined]
iswritable at .\iostream.jl:73 [inlined]
write at .\iostream.jl:368
writetag at C:\workdir\usr\share\julia\stdlib\v1.10\Serialization\src\Serialization.jl:145 [inlined]
serialize at C:\workdir\usr\share\julia\stdlib\v1.10\Serialization\src\Serialization.jl:622
serialize at C:\workdir\usr\share\julia\stdlib\v1.10\Serialization\src\Serialization.jl:268
serialize_dict_data at C:\workdir\usr\share\julia\stdlib\v1.10\Serialization\src\Serialization.jl:346
serialize at C:\workdir\usr\share\julia\stdlib\v1.10\Serialization\src\Serialization.jl:352
unknown function (ip: 0000014d2688b86f)
serialize_any at C:\workdir\usr\share\julia\stdlib\v1.10\Serialization\src\Serialization.jl:676
serialize at C:\workdir\usr\share\julia\stdlib\v1.10\Serialization\src\Serialization.jl:655
unknown function (ip: 0000014d299ea38f)
serialize_dict_data at C:\workdir\usr\share\julia\stdlib\v1.10\Serialization\src\Serialization.jl:346
serialize at C:\workdir\usr\share\julia\stdlib\v1.10\Serialization\src\Serialization.jl:352
unknown function (ip: 0000014d2688802f)
serialize at C:\workdir\usr\share\julia\stdlib\v1.10\Serialization\src\Serialization.jl:276
unknown function (ip: 0000014d26887aff)
serialize_any at C:\workdir\usr\share\julia\stdlib\v1.10\Serialization\src\Serialization.jl:676
serialize at C:\workdir\usr\share\julia\stdlib\v1.10\Serialization\src\Serialization.jl:655 [inlined]
serialize at C:\workdir\usr\share\julia\stdlib\v1.10\Serialization\src\Serialization.jl:777 [inlined]
#1 at C:\workdir\usr\share\julia\stdlib\v1.10\Serialization\src\Serialization.jl:788 [inlined]
#open#424 at .\io.jl:396
open at .\io.jl:393 [inlined]
serialize at C:\workdir\usr\share\julia\stdlib\v1.10\Serialization\src\Serialization.jl:788 [inlined]
volatilitypremiumsbacktest at C:\dev\Apollo\src\volatility_premiums\option_portfolios\vrp_backtest.jl:733
unknown function (ip: 0000014d29a24ef1)
jl_apply at C:/workdir/src\julia.h:1982 [inlined]
do_call at C:/workdir/src\interpreter.c:126
eval_value at C:/workdir/src\interpreter.c:223
eval_stmt_value at C:/workdir/src\interpreter.c:174 [inlined]
eval_body at C:/workdir/src\interpreter.c:635
jl_interpret_toplevel_thunk at C:/workdir/src\interpreter.c:775
jl_toplevel_eval_flex at C:/workdir/src\toplevel.c:934
jl_toplevel_eval_flex at C:/workdir/src\toplevel.c:877
jl_toplevel_eval_flex at C:/workdir/src\toplevel.c:877
eval_body at C:/workdir/src\interpreter.c:579
eval_body at C:/workdir/src\interpreter.c:544
jl_interpret_toplevel_thunk at C:/workdir/src\interpreter.c:775
jl_toplevel_eval_flex at C:/workdir/src\toplevel.c:934
ijl_toplevel_eval at C:/workdir/src\toplevel.c:943 [inlined]
ijl_toplevel_eval_in at C:/workdir/src\toplevel.c:985
eval at .\boot.jl:385 [inlined]
eval_user_input at C:\workdir\usr\share\julia\stdlib\v1.10\REPL\src\REPL.jl:150
repl_backend_loop at C:\workdir\usr\share\julia\stdlib\v1.10\REPL\src\REPL.jl:246
#start_repl_backend#46 at C:\workdir\usr\share\julia\stdlib\v1.10\REPL\src\REPL.jl:231
start_repl_backend at C:\workdir\usr\share\julia\stdlib\v1.10\REPL\src\REPL.jl:228
#run_repl#59 at C:\workdir\usr\share\julia\stdlib\v1.10\REPL\src\REPL.jl:389
run_repl at C:\workdir\usr\share\julia\stdlib\v1.10\REPL\src\REPL.jl:375
jfptr_run_repl_95862.1 at C:\Julia\Julia-1.10.5\lib\julia\sys.dll (unknown line)
#1013 at .\client.jl:432
jfptr_YY.1013_86638.1 at C:\Julia\Julia-1.10.5\lib\julia\sys.dll (unknown line)
jl_apply at C:/workdir/src\julia.h:1982 [inlined]
jl_f__call_latest at C:/workdir/src\builtins.c:812
#invokelatest#2 at .\essentials.jl:892 [inlined]
invokelatest at .\essentials.jl:889 [inlined]
run_main_repl at .\client.jl:416
exec_options at .\client.jl:333
_start at .\client.jl:552
jfptr__start_86663.1 at C:\Julia\Julia-1.10.5\lib\julia\sys.dll (unknown line)
jl_apply at C:/workdir/src\julia.h:1982 [inlined]
true_main at C:/workdir/src\jlapi.c:582
jl_repl_entrypoint at C:/workdir/src\jlapi.c:731
mainCRTStartup at C:/workdir/cli\loader_exe.c:58
BaseThreadInitThunk at C:\Windows\System32\KERNEL32.DLL (unknown line)
RtlUserThreadStart at C:\Windows\SYSTEM32\ntdll.dll (unknown line)
Allocations: 6483048742 (Pool: 6398019599; Big: 85029143); GC: 3353
2012-12-31T00:00:00: rebalance portfolio (expiry = 2013-01-07T00:00:00)
ERROR: OutOfMemoryError()
Stacktrace:
  [1] Array
    @ .\boot.jl:477 [inlined]
  [2] Array
    @ .\boot.jl:486 [inlined]
  [3] similar
    @ .\abstractarray.jl:877 [inlined]
  [4] similar
    @ .\abstractarray.jl:876 [inlined]
  [5] _array_for
    @ .\array.jl:723 [inlined]
  [6] _array_for
    @ .\array.jl:726 [inlined]
  [7] putvarboundlist(task::Mosek.Task, sub::Vector{Int32}, bkx::Vector{Mosek.Boundkey}, blx::Vector{Float64}, bux::Vector{Float64})
    @ Mosek C:\Users\userx\.julia\packages\Mosek\cMwUF\src\msk_functions.jl:16210
  [8] init_columns(task::Mosek.Task, cols::MosekTools.ColumnIndices)
    @ MosekTools C:\Users\userx\.julia\packages\MosekTools\ejym8\src\variable.jl:29
  [9] init_columns
    @ C:\Users\userx\.julia\packages\MosekTools\ejym8\src\variable.jl:39 [inlined]
 [10] add_variable(m::MosekTools.Optimizer)
    @ MosekTools C:\Users\userx\.julia\packages\MosekTools\ejym8\src\variable.jl:231
 [11] add_constrained_variable
    @ C:\Users\userx\.julia\packages\MathOptInterface\1fRdT\src\variables.jl:97 [inlined]
 [12] add_constrained_variable
    @ C:\Users\userx\.julia\packages\MathOptInterface\1fRdT\src\Bridges\bridge_optimizer.jl:2139 [inlined]
 [13] _add_variable_with_domain(dest::MathOptInterface.Bridges.LazyBridgeOptimizer{…}, src::MathOptInterface.Utilities.UniversalFallback{…}, index_map::MathOptInterface.Utilities.IndexMap, f::Vector{…}, ci::MathOptInterface.ConstraintIndex{…})
    @ MathOptInterface.Utilities C:\Users\userx\.julia\packages\MathOptInterface\1fRdT\src\Utilities\copy.jl:504
 [14] _copy_variables_with_set(dest::MathOptInterface.Bridges.LazyBridgeOptimizer{MosekTools.Optimizer}, src::MathOptInterface.Utilities.UniversalFallback{MathOptInterface.Utilities.Model{Float64}})
    @ MathOptInterface.Utilities C:\Users\userx\.julia\packages\MathOptInterface\1fRdT\src\Utilities\copy.jl:548
 [15] default_copy_to(dest::MathOptInterface.Bridges.LazyBridgeOptimizer{MosekTools.Optimizer}, src::MathOptInterface.Utilities.UniversalFallback{MathOptInterface.Utilities.Model{Float64}})
    @ MathOptInterface.Utilities C:\Users\userx\.julia\packages\MathOptInterface\1fRdT\src\Utilities\copy.jl:386
 [16] copy_to
    @ C:\Users\userx\.julia\packages\MathOptInterface\1fRdT\src\Bridges\bridge_optimizer.jl:442 [inlined]
 [17] optimize!
    @ C:\Users\userx\.julia\packages\MathOptInterface\1fRdT\src\MathOptInterface.jl:121 [inlined]
 [18] optimize!(m::MathOptInterface.Utilities.CachingOptimizer{MathOptInterface.Bridges.LazyBridgeOptimizer{MosekTools.Optimizer}, MathOptInterface.Utilities.UniversalFallback{MathOptInterface.Utilities.Model{Float64}}})
    @ MathOptInterface.Utilities C:\Users\userx\.julia\packages\MathOptInterface\1fRdT\src\Utilities\cachingoptimizer.jl:321
 [19] optimize!(model::JuMP.Model; ignore_optimize_hook::Bool, _differentiation_backend::MathOptInterface.Nonlinear.SparseReverseMode, kwargs::@Kwargs{})
    @ JuMP C:\Users\userx\.julia\packages\JuMP\PKxEW\src\optimizer_interface.jl:595
 [20] optimize!
    @ C:\Users\userx\.julia\packages\JuMP\PKxEW\src\optimizer_interface.jl:546 [inlined]
 [21] optimizecrrautility(π::Vector{Float64}, R::LinearAlgebra.Adjoint{Float64, Matrix{Float64}}, rf::Float64, γ::Float64, W₀::Float64, structures::DataFrames.DataFrame)
    @ Apollo C:\dev\Apollo\src\volatility_premiums\option_portfolios\vrp_backtest.jl:290
 [22] positiveskewedportfolio(π::Vector{Float64}, scenarios::LinearAlgebra.Adjoint{Float64, Matrix{Float64}}, rf::Float64, initial_wealth::Float64, structures::DataFrames.DataFrame)
    @ Apollo C:\dev\Apollo\src\volatility_premiums\option_portfolios\vrp_backtest.jl:421
 [23] optimalportfolio(method::String, π::Vector{Float64}, R::LinearAlgebra.Adjoint{Float64, Matrix{Float64}}, rf::Float64, W₀::Float64, structures::DataFrames.DataFrame)
    @ Apollo C:\dev\Apollo\src\volatility_premiums\option_portfolios\vrp_backtest.jl:453
 [24] volatilitypremiumsbacktest(tenor::String, initial_wealth::Float64, file_post_fix::String, optimization_method::String)
    @ Apollo C:\dev\Apollo\src\volatility_premiums\option_portfolios\vrp_backtest.jl:627
 [25] top-level scope
    @ REPL[8]:1
 [26] top-level scope
    @ none:1
Some type information was truncated. Use `show(err)` to see complete types.
Please submit a bug report with steps to reproduce this fault, and any error messages that follow (in their entirety). Thanks.
Exception: EXCEPTION_ACCESS_VIOLATION at 0x1c73c6f5345 -- setindex! at .\array.jl:1021 [inlined]
macro expansion at .\broadcast.jl:1004 [inlined]
macro expansion at .\simdloop.jl:77 [inlined]
copyto! at .\broadcast.jl:1003 [inlined]
copyto! at .\broadcast.jl:956 [inlined]
copy at .\broadcast.jl:928 [inlined]
materialize at .\broadcast.jl:903 [inlined]
#168 at C:\dev\Apollo\src\volatility_premiums\option_portfolios\fhs_densities.jl:51
in expression starting at REPL[7]:1
setindex! at .\array.jl:1021 [inlined]
macro expansion at .\broadcast.jl:1004 [inlined]
macro expansion at .\simdloop.jl:77 [inlined]
copyto! at .\broadcast.jl:1003 [inlined]
copyto! at .\broadcast.jl:956 [inlined]
copy at .\broadcast.jl:928 [inlined]
materialize at .\broadcast.jl:903 [inlined]
#168 at C:\dev\Apollo\src\volatility_premiums\option_portfolios\fhs_densities.jl:51
_inner_mapslices! at .\abstractarray.jl:3248
unknown function (ip: 000001c73c6f7ad4)
#mapslices#199 at .\abstractarray.jl:3236
unknown function (ip: 000001c73c6f6296)
mapslices at .\abstractarray.jl:3175
unknown function (ip: 000001c73c6f35b3)
armagarchscenerios at C:\dev\Apollo\src\volatility_premiums\option_portfolios\fhs_densities.jl:51
unknown function (ip: 000001c73c6d6583)
generatepaths at C:\dev\Apollo\src\volatility_premiums\option_portfolios\op_utils.jl:6
unknown function (ip: 000001c73c6c6190)
volatilitypremiumsbacktest at C:\dev\Apollo\src\volatility_premiums\option_portfolios\vrp_backtest.jl:609
unknown function (ip: 000001c71b0f3a51)
jl_apply at C:/workdir/src\julia.h:1982 [inlined]
do_call at C:/workdir/src\interpreter.c:126
eval_value at C:/workdir/src\interpreter.c:223
eval_stmt_value at C:/workdir/src\interpreter.c:174 [inlined]
eval_body at C:/workdir/src\interpreter.c:635
jl_interpret_toplevel_thunk at C:/workdir/src\interpreter.c:775
jl_toplevel_eval_flex at C:/workdir/src\toplevel.c:934
jl_toplevel_eval_flex at C:/workdir/src\toplevel.c:877
jl_toplevel_eval_flex at C:/workdir/src\toplevel.c:877
eval_body at C:/workdir/src\interpreter.c:579
eval_body at C:/workdir/src\interpreter.c:544
jl_interpret_toplevel_thunk at C:/workdir/src\interpreter.c:775
jl_toplevel_eval_flex at C:/workdir/src\toplevel.c:934
ijl_toplevel_eval at C:/workdir/src\toplevel.c:943 [inlined]
ijl_toplevel_eval_in at C:/workdir/src\toplevel.c:985
eval at .\boot.jl:385 [inlined]
eval_user_input at C:\workdir\usr\share\julia\stdlib\v1.10\REPL\src\REPL.jl:150
repl_backend_loop at C:\workdir\usr\share\julia\stdlib\v1.10\REPL\src\REPL.jl:246
#start_repl_backend#46 at C:\workdir\usr\share\julia\stdlib\v1.10\REPL\src\REPL.jl:231
start_repl_backend at C:\workdir\usr\share\julia\stdlib\v1.10\REPL\src\REPL.jl:228
#run_repl#59 at C:\workdir\usr\share\julia\stdlib\v1.10\REPL\src\REPL.jl:389
run_repl at C:\workdir\usr\share\julia\stdlib\v1.10\REPL\src\REPL.jl:375
jfptr_run_repl_95862.1 at C:\Julia\Julia-1.10.5\lib\julia\sys.dll (unknown line)
#1013 at .\client.jl:432
jfptr_YY.1013_86638.1 at C:\Julia\Julia-1.10.5\lib\julia\sys.dll (unknown line)
jl_apply at C:/workdir/src\julia.h:1982 [inlined]
jl_f__call_latest at C:/workdir/src\builtins.c:812
#invokelatest#2 at .\essentials.jl:892 [inlined]
invokelatest at .\essentials.jl:889 [inlined]
run_main_repl at .\client.jl:416
exec_options at .\client.jl:333
_start at .\client.jl:552
jfptr__start_86663.1 at C:\Julia\Julia-1.10.5\lib\julia\sys.dll (unknown line)
jl_apply at C:/workdir/src\julia.h:1982 [inlined]
true_main at C:/workdir/src\jlapi.c:582
jl_repl_entrypoint at C:/workdir/src\jlapi.c:731
mainCRTStartup at C:/workdir/cli\loader_exe.c:58
BaseThreadInitThunk at C:\Windows\System32\KERNEL32.DLL (unknown line)
RtlUserThreadStart at C:\Windows\SYSTEM32\ntdll.dll (unknown line)
Allocations: 569707970 (Pool: 569531362; Big: 176608); GC: 727
Another ERROR
ERROR: AssertionError: bk == MSK_BK_UP
Stacktrace:
  [1] add_variable_constraint
    @ C:\Users\userx\.julia\packages\MosekTools\ejym8\src\constraint.jl:271 [inlined]
  [2] add_constraint(m::MosekTools.Optimizer, xs::MathOptInterface.VariableIndex, dom::MathOptInterface.GreaterThan{Float64})
    @ MosekTools C:\Users\userx\.julia\packages\MosekTools\ejym8\src\constraint.jl:466
  [3] add_constrained_variable
    @ C:\Users\userx\.julia\packages\MathOptInterface\1fRdT\src\variables.jl:98 [inlined]
  [4] add_constrained_variable
    @ C:\Users\userx\.julia\packages\MathOptInterface\1fRdT\src\Bridges\bridge_optimizer.jl:2139 [inlined]
  [5] _add_variable_with_domain(dest::MathOptInterface.Bridges.LazyBridgeOptimizer{…}, src::MathOptInterface.Utilities.UniversalFallback{…}, index_map::MathOptInterface.Utilities.IndexMap, f::Vector{…}, ci::MathOptInterface.ConstraintIndex{…})
    @ MathOptInterface.Utilities C:\Users\userx\.julia\packages\MathOptInterface\1fRdT\src\Utilities\copy.jl:504
  [6] _copy_variables_with_set(dest::MathOptInterface.Bridges.LazyBridgeOptimizer{…}, src::MathOptInterface.Utilities.UniversalFallback{…})
    @ MathOptInterface.Utilities C:\Users\userx\.julia\packages\MathOptInterface\1fRdT\src\Utilities\copy.jl:548
  [7] default_copy_to(dest::MathOptInterface.Bridges.LazyBridgeOptimizer{…}, src::MathOptInterface.Utilities.UniversalFallback{…})
    @ MathOptInterface.Utilities C:\Users\userx\.julia\packages\MathOptInterface\1fRdT\src\Utilities\copy.jl:386
  [8] copy_to
    @ C:\Users\userx\.julia\packages\MathOptInterface\1fRdT\src\Bridges\bridge_optimizer.jl:442 [inlined]
  [9] optimize!
    @ C:\Users\userx\.julia\packages\MathOptInterface\1fRdT\src\MathOptInterface.jl:121 [inlined]
 [10] optimize!(m::MathOptInterface.Utilities.CachingOptimizer{…})
    @ MathOptInterface.Utilities C:\Users\userx\.julia\packages\MathOptInterface\1fRdT\src\Utilities\cachingoptimizer.jl:321
 [11] optimize!(model::JuMP.Model; ignore_optimize_hook::Bool, _differentiation_backend::MathOptInterface.Nonlinear.SparseReverseMode, kwargs::@Kwargs{})
    @ JuMP C:\Users\userx\.julia\packages\JuMP\PKxEW\src\optimizer_interface.jl:595
 [12] optimize!
    @ C:\Users\userx\.julia\packages\JuMP\PKxEW\src\optimizer_interface.jl:546 [inlined]
 [13] optimizecrrautility(π::Vector{…}, R::LinearAlgebra.Adjoint{…}, rf::Float64, γ::Float64, W₀::Float64, structures::DataFrame)
    @ Apollo c:\dev\Apollo\src\volatility_premiums\option_portfolios\vrp_backtest.jl:290
 [14] positiveskewedportfolio(π::Vector{…}, scenarios::LinearAlgebra.Adjoint{…}, rf::Float64, initial_wealth::Float64, structures::DataFrame)
    @ Apollo c:\dev\Apollo\src\volatility_premiums\option_portfolios\vrp_backtest.jl:421
 [15] optimalportfolio(method::String, π::Vector{…}, R::LinearAlgebra.Adjoint{…}, rf::Float64, W₀::Float64, structures::DataFrame)
    @ Apollo c:\dev\Apollo\src\volatility_premiums\option_portfolios\vrp_backtest.jl:453
 [16] volatilitypremiumsbacktest(tenor::String, initial_wealth::Float64, file_post_fix::String, optimization_method::String)
    @ Apollo c:\dev\Apollo\src\volatility_premiums\option_portfolios\vrp_backtest.jl:627
 [17] top-level scope
    @ c:\dev\Apollo\scracthpad\vol_premiums_backtest.jl:1
Some type information was truncated. Use `show(err)` to see complete types.

Here's the output from versioninfo()

julia> versioninfo()
Julia Version 1.10.5
Commit 6f3fdf7b36 (2024-08-27 14:19 UTC)
Build Info:
  Official https://julialang.org/ release
Platform Info:
  OS: Windows (x86_64-w64-mingw32)
  CPU: 32 × 13th Gen Intel(R) Core(TM) i9-13900K
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-15.0.7 (ORCJIT, goldmont)
Threads: 8 default, 0 interactive, 4 GC (on 32 virtual cores)
Environment:
  JULIA_EDITOR = code
  JULIA_NUM_THREADS = 8
AUK1939 commented 1 week ago

On further investigation seems like the program runs if I set JULIA_NUM_THREADS = 1. Note I don't expricitly use any threading features in my program.

giordano commented 1 week ago

You haven't provided a minimal reproducible example to reproduce your issue. Also, the error messages you posted are a bit confusing as they reference multiple things (there seems to be also one involving R?) and the error

Another ERROR
ERROR: AssertionError: bk == MSK_BK_UP

seems to be related to a package you're using MosekTools. Please, provide more information about how this is a bug in Julia itself and not a third-party package (in which case you should report the bug there), otherwise this ticket will be closed at some point.

AUK1939 commented 1 week ago

You're right - there's a lot going on here. I'll try and break this down the old fashioned way (logs, process of elimination) and try and create something reproducible. So far I've established that somethings up with the serialization call and threading.

mbauman commented 1 week ago
Exception: EXCEPTION_ACCESS_VIOLATION at 0x1c73c6f5345 -- setindex! at .\array.jl:1021 [inlined]

Given this line, julia --check-bounds=yes would be a very good start. It's not unlikely that all these crashes are stemming from an OOB write, it's just that it sometimes ends up silently corrupting internal state instead of reaching outside the program segment and causing an immediate segfault.

mbauman commented 1 week ago

Ah, but that actually means this is more likely to be an implementation/package problem than a Julia one. My comment was just a pointer on how you might start finding the root cause to build out that minimal working example — once you have more details you can reopen with those.

AUK1939 commented 1 week ago

Thanks @mbauman, thanks for your help. I'm running the code with julia --check-bounds=yes. Let me know if you have any other useful tips to help pin this down, and help create a MWE