SciML / JumpProcesses.jl

Build and simulate jump equations like Gillespie simulations and jump diffusions with constant and state-dependent rates and mix with differential equations and scientific machine learning (SciML)
https://docs.sciml.ai/JumpProcesses/stable/
Other
140 stars 35 forks source link

BenchmarkTools + DiffEqJump Bug #205

Closed Vilin97 closed 3 years ago

Vilin97 commented 3 years ago

Update: I was able to get the same error without a priority queue. Script:

using DiffEqJump, Random, BenchmarkTools, DataStructures
const MINJUMPRATE = 2.0^exponent(1e-12)

function pq_setup(rates)
    pqdata = randexp(length(rates))./rates
    MutableBinaryMinHeap(pqdata)
end

function pt_setup!(pt, rates)
    DiffEqJump.reset!(pt)
    for (pid,priority) in enumerate(rates)
        DiffEqJump.insert!(pt, pid, priority)
    end
end

# setup for PT
minrate = MINJUMPRATE
minexponent = exponent(minrate)
minrate = 2.0^minexponent
ratetogroup = rate -> DiffEqJump.priortogid(rate, minexponent)
pt = DiffEqJump.PriorityTable(ratetogroup, zeros(1), minrate, 2*minrate)

# benchmark
num_rates = 100
pt_setup!(pt, rand(num_rates))

b = @benchmarkable pt_setup!(pt, rates) setup = (rates = rand(num_rates))
run(b)

rates = rand(num_rates)
b = @benchmarkable DiffEqJump.update!(pt, n, oldrate, rates[n]) setup = (n = rand(1:num_rates); oldrate = rates[n]; rates[n] = rand())
run(b)

Error:

julia> include("src\\benchmarks\\structs.jl")

Please submit a bug report with steps to reproduce this fault, and any error messages that follow (in their entirety). Thanks.
Exception: EXCEPTION_ACCESS_VIOLATION at 0x2775a32 -- gc_try_setmark at /cygdrive/c/buildbot/worker/package_win64/build/src\gc.c:1704 [inlined]
gc_mark_scan_objarray at /cygdrive/c/buildbot/worker/package_win64/build/src\gc.c:1816 [inlined]
gc_mark_loop at /cygdrive/c/buildbot/worker/package_win64/build/src\gc.c:2207
in expression starting at C:\Users\Vasily\.julia\dev\DiffEqJump\src\benchmarks\structs.jl:29
gc_try_setmark at /cygdrive/c/buildbot/worker/package_win64/build/src\gc.c:1704 [inlined]
gc_mark_scan_objarray at /cygdrive/c/buildbot/worker/package_win64/build/src\gc.c:1816 [inlined]
gc_mark_loop at /cygdrive/c/buildbot/worker/package_win64/build/src\gc.c:2207
_jl_gc_collect at /cygdrive/c/buildbot/worker/package_win64/build/src\gc.c:3034
jl_gc_collect at /cygdrive/c/buildbot/worker/package_win64/build/src\gc.c:3241
gc at .\gcutils.jl:94 [inlined]
gc at .\gcutils.jl:94 [inlined]
gcscrub at C:\Users\Vasily\.julia\packages\BenchmarkTools\V1XzU\src\execution.jl:4 [inlined]
#_run#48 at C:\Users\Vasily\.julia\packages\BenchmarkTools\V1XzU\src\execution.jl:94
_run at C:\Users\Vasily\.julia\packages\BenchmarkTools\V1XzU\src\execution.jl:92
jl_apply at /cygdrive/c/buildbot/worker/package_win64/build/src\julia.h:1703 [inlined]
jl_f__call_latest at /cygdrive/c/buildbot/worker/package_win64/build/src\builtins.c:714
#invokelatest#2 at .\essentials.jl:708 [inlined]
invokelatest at .\essentials.jl:706 [inlined]
#run_result#45 at C:\Users\Vasily\.julia\packages\BenchmarkTools\V1XzU\src\execution.jl:33 [inlined]
run_result at C:\Users\Vasily\.julia\packages\BenchmarkTools\V1XzU\src\execution.jl:33 [inlined]
#run#49 at C:\Users\Vasily\.julia\packages\BenchmarkTools\V1XzU\src\execution.jl:116
run at C:\Users\Vasily\.julia\packages\BenchmarkTools\V1XzU\src\execution.jl:116 [inlined]
run at C:\Users\Vasily\.julia\packages\BenchmarkTools\V1XzU\src\execution.jl:116
jl_apply at /cygdrive/c/buildbot/worker/package_win64/build/src\julia.h:1703 [inlined]
do_call at /cygdrive/c/buildbot/worker/package_win64/build/src\interpreter.c:115
eval_value at /cygdrive/c/buildbot/worker/package_win64/build/src\interpreter.c:204
eval_stmt_value at /cygdrive/c/buildbot/worker/package_win64/build/src\interpreter.c:155 [inlined]eval_body at /cygdrive/c/buildbot/worker/package_win64/build/src\interpreter.c:576
jl_interpret_toplevel_thunk at /cygdrive/c/buildbot/worker/package_win64/build/src\interpreter.c:670
jl_toplevel_eval_flex at /cygdrive/c/buildbot/worker/package_win64/build/src\toplevel.c:877
jl_toplevel_eval_flex at /cygdrive/c/buildbot/worker/package_win64/build/src\toplevel.c:825
jl_toplevel_eval at /cygdrive/c/buildbot/worker/package_win64/build/src\toplevel.c:886 [inlined]
jl_toplevel_eval_in at /cygdrive/c/buildbot/worker/package_win64/build/src\toplevel.c:929
eval at .\boot.jl:360 [inlined]
include_string at .\loading.jl:1094
_include at .\loading.jl:1148
include at .\client.jl:444
jl_apply at /cygdrive/c/buildbot/worker/package_win64/build/src\julia.h:1703 [inlined]
do_call at /cygdrive/c/buildbot/worker/package_win64/build/src\interpreter.c:115
eval_value at /cygdrive/c/buildbot/worker/package_win64/build/src\interpreter.c:204
eval_stmt_value at /cygdrive/c/buildbot/worker/package_win64/build/src\interpreter.c:155 [inlined]eval_body at /cygdrive/c/buildbot/worker/package_win64/build/src\interpreter.c:576
jl_interpret_toplevel_thunk at /cygdrive/c/buildbot/worker/package_win64/build/src\interpreter.c:670
jl_toplevel_eval_flex at /cygdrive/c/buildbot/worker/package_win64/build/src\toplevel.c:877
jl_toplevel_eval_flex at /cygdrive/c/buildbot/worker/package_win64/build/src\toplevel.c:825
jl_toplevel_eval at /cygdrive/c/buildbot/worker/package_win64/build/src\toplevel.c:886 [inlined]
jl_toplevel_eval_in at /cygdrive/c/buildbot/worker/package_win64/build/src\toplevel.c:929
eval at .\boot.jl:360 [inlined]
eval_user_input at C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.6\REPL\src\REPL.jl:139
repl_backend_loop at C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.6\REPL\src\REPL.jl:200
start_repl_backend at C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.6\REPL\src\REPL.jl:185
#run_repl#42 at C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.6\REPL\src\REPL.jl:317
run_repl at C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.6\REPL\src\REPL.jl:305
#874 at .\client.jl:387
jfptr_YY.874_40889.clone_1 at C:\Users\Vasily\AppData\Local\Programs\Julia\Julia-1.6.1\lib\julia\sys.dll (unknown line)
jl_apply at /cygdrive/c/buildbot/worker/package_win64/build/src\julia.h:1703 [inlined]
jl_f__call_latest at /cygdrive/c/buildbot/worker/package_win64/build/src\builtins.c:714
#invokelatest#2 at .\essentials.jl:708 [inlined]
invokelatest at .\essentials.jl:706 [inlined]
run_main_repl at .\client.jl:372
exec_options at .\client.jl:302
_start at .\client.jl:485
jfptr__start_22922.clone_1 at C:\Users\Vasily\AppData\Local\Programs\Julia\Julia-1.6.1\lib\julia\sys.dll (unknown line)
jl_apply at /cygdrive/c/buildbot/worker/package_win64/build/src\julia.h:1703 [inlined]
true_main at /cygdrive/c/buildbot/worker/package_win64/build/src\jlapi.c:560
repl_entrypoint at /cygdrive/c/buildbot/worker/package_win64/build/src\jlapi.c:702
mainCRTStartup at /cygdrive/c/buildbot/worker/package_win64/build/cli\loader_exe.c:51
BaseThreadInitThunk at C:\WINDOWS\System32\KERNEL32.DLL (unknown line)
RtlUserThreadStart at C:\WINDOWS\SYSTEM32\ntdll.dll (unknown line)
Allocations: 58082045 (Pool: 58075403; Big: 6642); GC: 94

End of Update.

Running the following script produces an error. Script:

using DiffEqJump, Random, BenchmarkTools, DataStructures
const MINJUMPRATE = 2.0^exponent(1e-12)

num_rates = 100
num_updates = 1e6
rates = rand(num_rates)

# PQ setup
pqdata = randexp(num_rates)./rates
pq = MutableBinaryMinHeap(pqdata)

# PT setup
minrate = MINJUMPRATE
minexponent = exponent(minrate)
minrate = 2.0^minexponent
ratetogroup = rate -> DiffEqJump.priortogid(rate, minexponent)
rt = DiffEqJump.PriorityTable(ratetogroup, zeros(1), minrate, 2*minrate)

DiffEqJump.reset!(rt)
for (pid,priority) in enumerate(rates)
    DiffEqJump.insert!(rt, pid, priority)
end

# benchmark updating
println("benchmarking")
b1 = @benchmarkable update!(pq, n, randexp() / rates[n]) setup = (n = rand(1:num_rates); rates[n] = rand())
bench1 = run(b1)

b2 = @benchmarkable DiffEqJump.update!(rt, n, oldrate, rates[n]) setup = (n = rand(1:num_rates); oldrate = rates[n]; rates[n] = rand())
run(b2)

Now run it:

julia> include("src\\benchmarks\\structs.jl")
benchmarking

Please submit a bug report with steps to reproduce this fault, and any error messages that follow (in their entirety). Thanks.
Exception: EXCEPTION_ACCESS_VIOLATION at 0x26fadd9 -- jl_gc_pool_alloc at /cygdrive/c/buildbot/worker/package_win64/build/src\gc.c:1210 [inlined]
jl_gc_alloc_ at /cygdrive/c/buildbot/worker/package_win64/build/src\julia_internal.h:285 [inlined]jl_gc_alloc at /cygdrive/c/buildbot/worker/package_win64/build/src\gc.c:3283
in expression starting at none:0
jl_gc_pool_alloc at /cygdrive/c/buildbot/worker/package_win64/build/src\gc.c:1210 [inlined]
jl_gc_alloc_ at /cygdrive/c/buildbot/worker/package_win64/build/src\julia_internal.h:285 [inlined]jl_gc_alloc at /cygdrive/c/buildbot/worker/package_win64/build/src\gc.c:3283
_new_array_ at /cygdrive/c/buildbot/worker/package_win64/build/src\array.c:122
_new_array at /cygdrive/c/buildbot/worker/package_win64/build/src\array.c:188 [inlined]
jl_alloc_array_1d at /cygdrive/c/buildbot/worker/package_win64/build/src\array.c:459
decode_backtrace at /cygdrive/c/buildbot/worker/package_win64/build/src\stackwalk.c:298
jl_get_excstack at /cygdrive/c/buildbot/worker/package_win64/build/src\stackwalk.c:353
#catch_stack#61 at .\error.jl:144
catch_stack at .\error.jl:144 [inlined]
catch_stack at .\error.jl:144 [inlined]
eval_user_input at C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.6\REPL\src\REPL.jl:151
repl_backend_loop at C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.6\REPL\src\REPL.jl:200
start_repl_backend at C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.6\REPL\src\REPL.jl:185
#run_repl#42 at C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.6\REPL\src\REPL.jl:317
run_repl at C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.6\REPL\src\REPL.jl:305
#874 at .\client.jl:387
jfptr_YY.874_40889.clone_1 at C:\Users\Vasily\AppData\Local\Programs\Julia\Julia-1.6.1\lib\julia\sys.dll (unknown line)
jl_apply at /cygdrive/c/buildbot/worker/package_win64/build/src\julia.h:1703 [inlined]
jl_f__call_latest at /cygdrive/c/buildbot/worker/package_win64/build/src\builtins.c:714
#invokelatest#2 at .\essentials.jl:708 [inlined]
invokelatest at .\essentials.jl:706 [inlined]
run_main_repl at .\client.jl:372
exec_options at .\client.jl:302
_start at .\client.jl:485
jfptr__start_22922.clone_1 at C:\Users\Vasily\AppData\Local\Programs\Julia\Julia-1.6.1\lib\julia\sys.dll (unknown line)
jl_apply at /cygdrive/c/buildbot/worker/package_win64/build/src\julia.h:1703 [inlined]
true_main at /cygdrive/c/buildbot/worker/package_win64/build/src\jlapi.c:560
repl_entrypoint at /cygdrive/c/buildbot/worker/package_win64/build/src\jlapi.c:702
mainCRTStartup at /cygdrive/c/buildbot/worker/package_win64/build/cli\loader_exe.c:51
BaseThreadInitThunk at C:\WINDOWS\System32\KERNEL32.DLL (unknown line)
RtlUserThreadStart at C:\WINDOWS\SYSTEM32\ntdll.dll (unknown line)
Allocations: 18457956 (Pool: 18453504; Big: 4452); GC: 21

Please submit a bug report with steps to reproduce this fault, and any error messages that follow (in their entirety). Thanks.
Exception: EXCEPTION_ACCESS_VIOLATION at 0x26fa605 -- jl_gc_pool_alloc at /cygdrive/c/buildbot/worker/package_win64/build/src\gc.c:1210
in expression starting at none:0
jl_gc_pool_alloc at /cygdrive/c/buildbot/worker/package_win64/build/src\gc.c:1210
jl_gc_alloc_ at /cygdrive/c/buildbot/worker/package_win64/build/src\julia_internal.h:285 [inlined]jl_typemap_alloc at /cygdrive/c/buildbot/worker/package_win64/build/src\typemap.c:1271
cache_method at /cygdrive/c/buildbot/worker/package_win64/build/src\gf.c:1139
ml_matches at /cygdrive/c/buildbot/worker/package_win64/build/src\gf.c:3121
_gf_invoke_lookup at /cygdrive/c/buildbot/worker/package_win64/build/src\gf.c:2430
jl_mt_assoc_by_type at /cygdrive/c/buildbot/worker/package_win64/build/src\gf.c:1188 [inlined]
jl_mt_assoc_by_type at /cygdrive/c/buildbot/worker/package_win64/build/src\gf.c:1170
jl_lookup_generic_ at /cygdrive/c/buildbot/worker/package_win64/build/src\gf.c:2390
jl_apply_generic at /cygdrive/c/buildbot/worker/package_win64/build/src\gf.c:2415
jl_apply at /cygdrive/c/buildbot/worker/package_win64/build/src\julia.h:1703 [inlined]
run_finalizer at /cygdrive/c/buildbot/worker/package_win64/build/src\gc.c:278
jl_gc_run_finalizers_in_list at /cygdrive/c/buildbot/worker/package_win64/build/src\gc.c:365
run_finalizers at /cygdrive/c/buildbot/worker/package_win64/build/src\gc.c:394 [inlined]
run_finalizers at /cygdrive/c/buildbot/worker/package_win64/build/src\gc.c:372
jl_atexit_hook at /cygdrive/c/buildbot/worker/package_win64/build/src\init.c:240
jl_exit at /cygdrive/c/buildbot/worker/package_win64/build/src\jl_uv.c:632
jl_exception_handler at /cygdrive/c/buildbot/worker/package_win64/build/src\signals-win.c:318
__julia_personality at /cygdrive/c/buildbot/worker/package_win64/build/src/support\win32_ucontext.c:28
_chkstk at C:\WINDOWS\SYSTEM32\ntdll.dll (unknown line)
RtlRaiseException at C:\WINDOWS\SYSTEM32\ntdll.dll (unknown line)
KiUserExceptionDispatcher at C:\WINDOWS\SYSTEM32\ntdll.dll (unknown line)
jl_gc_pool_alloc at /cygdrive/c/buildbot/worker/package_win64/build/src\gc.c:1209 [inlined]
jl_gc_alloc_ at /cygdrive/c/buildbot/worker/package_win64/build/src\julia_internal.h:285 [inlined]jl_gc_alloc at /cygdrive/c/buildbot/worker/package_win64/build/src\gc.c:3283
_new_array_ at /cygdrive/c/buildbot/worker/package_win64/build/src\array.c:122
_new_array at /cygdrive/c/buildbot/worker/package_win64/build/src\array.c:188 [inlined]
jl_alloc_array_1d at /cygdrive/c/buildbot/worker/package_win64/build/src\array.c:459
decode_backtrace at /cygdrive/c/buildbot/worker/package_win64/build/src\stackwalk.c:298
jl_get_excstack at /cygdrive/c/buildbot/worker/package_win64/build/src\stackwalk.c:353
#catch_stack#61 at .\error.jl:144
catch_stack at .\error.jl:144 [inlined]
catch_stack at .\error.jl:144 [inlined]
eval_user_input at C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.6\REPL\src\REPL.jl:151
repl_backend_loop at C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.6\REPL\src\REPL.jl:200
start_repl_backend at C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.6\REPL\src\REPL.jl:185
#run_repl#42 at C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.6\REPL\src\REPL.jl:317
run_repl at C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.6\REPL\src\REPL.jl:305
#874 at .\client.jl:387
jfptr_YY.874_40889.clone_1 at C:\Users\Vasily\AppData\Local\Programs\Julia\Julia-1.6.1\lib\julia\sys.dll (unknown line)
jl_apply at /cygdrive/c/buildbot/worker/package_win64/build/src\julia.h:1703 [inlined]
jl_f__call_latest at /cygdrive/c/buildbot/worker/package_win64/build/src\builtins.c:714
#invokelatest#2 at .\essentials.jl:708 [inlined]
invokelatest at .\essentials.jl:706 [inlined]
run_main_repl at .\client.jl:372
exec_options at .\client.jl:302
_start at .\client.jl:485
jfptr__start_22922.clone_1 at C:\Users\Vasily\AppData\Local\Programs\Julia\Julia-1.6.1\lib\julia\sys.dll (unknown line)
jl_apply at /cygdrive/c/buildbot/worker/package_win64/build/src\julia.h:1703 [inlined]
true_main at /cygdrive/c/buildbot/worker/package_win64/build/src\jlapi.c:560
repl_entrypoint at /cygdrive/c/buildbot/worker/package_win64/build/src\jlapi.c:702
mainCRTStartup at /cygdrive/c/buildbot/worker/package_win64/build/cli\loader_exe.c:51
BaseThreadInitThunk at C:\WINDOWS\System32\KERNEL32.DLL (unknown line)
RtlUserThreadStart at C:\WINDOWS\SYSTEM32\ntdll.dll (unknown line)
Allocations: 18457956 (Pool: 18453504; Big: 4452); GC: 21

Observations:

  1. This error does not happen if bench1 = run(b1) is commented out.

  2. This error does not happen if run(b2) is evaluated before run(b1)

  3. Sometimes the error take the following form:

    ERROR: BoundsError: attempt to access 100-element Vector{Tuple{Int64, Int64}} at index [1320009731]
    Stacktrace:
    [1] setindex!(A::Vector{Tuple{Int64, Int64}}, x::Tuple{Int64, Int64}, i1::Int64)
    @ Base .\array.jl:839
    [2] update!(pt::DiffEqJump.PriorityTable{Float64, Int64, Int64, var"#1#2"}, pid::Int64, oldpriority::Float64, newpriority::Float64)
    @ DiffEqJump C:\Users\Vasily\.julia\dev\DiffEqJump\src\aggregators\prioritytable.jl:214
    [3] var"##core#257"(n::Int64, oldrate::Float64)
    @ Main C:\Users\Vasily\.julia\packages\BenchmarkTools\V1XzU\src\execution.jl:479
    [4] var"##sample#258"(__params::BenchmarkTools.Parameters)
    @ Main C:\Users\Vasily\.julia\packages\BenchmarkTools\V1XzU\src\execution.jl:485
    [5] _run(b::BenchmarkTools.Benchmark, p::BenchmarkTools.Parameters; verbose::Bool, pad::String, kwargs::Base.Iterators.Pairs{Union{}, Union{}, Tuple{}, NamedTuple{(), Tuple{}}})
    @ BenchmarkTools C:\Users\Vasily\.julia\packages\BenchmarkTools\V1XzU\src\execution.jl:104
    [6] _run(b::BenchmarkTools.Benchmark, p::BenchmarkTools.Parameters)
    @ BenchmarkTools C:\Users\Vasily\.julia\packages\BenchmarkTools\V1XzU\src\execution.jl:92
    [7] #invokelatest#2
    @ .\essentials.jl:708 [inlined]
    [8] invokelatest
    @ .\essentials.jl:706 [inlined]
    [9] #run_result#45
    @ C:\Users\Vasily\.julia\packages\BenchmarkTools\V1XzU\src\execution.jl:33 [inlined]
    [10] run_result
    @ C:\Users\Vasily\.julia\packages\BenchmarkTools\V1XzU\src\execution.jl:33 [inlined]
    [11] run(b::BenchmarkTools.Benchmark, p::BenchmarkTools.Parameters; progressid::Nothing, nleaves::Float64, ndone::Float64, kwargs::Base.Iterators.Pairs{Union{}, Union{}, Tuple{}, NamedTuple{(), Tuple{}}})
    @ BenchmarkTools C:\Users\Vasily\.julia\packages\BenchmarkTools\V1XzU\src\execution.jl:116
    [12] run (repeats 2 times)
    @ C:\Users\Vasily\.julia\packages\BenchmarkTools\V1XzU\src\execution.jl:116 [inlined]
    [13] top-level scope
    @ REPL[3]:1

    without Julia crashing.

  4. No error appears if I simply do

    for i in 1:10^6
    n = rand(1:num_rates); oldrate = rates[n]; rates[n] = rand()
    DiffEqJump.update!(rt, n, oldrate, rates[n])
    end

    instead of run(b2).

  5. No error appears if I do b1 = @benchmarkable DiffEqJump.update!(rt, n, oldrate, rates[n]) setup = (n = rand(1:num_rates); oldrate = rates[n]; rates[n] = rand()) instead of b1 = @benchmarkable update!(pq, n, randexp() / rates[n]) setup = (n = rand(1:num_rates); rates[n] = rand()).

  6. The error persists if I evaluate b1 = @benchmarkable sort([1,2,3]); run(b1) before doing run(b2).

  7. No error appears if I do b1 = @benchmarkable update!(pq, 1, randexp() / rates[1]) instead of b1 = @benchmarkable update!(pq, n, randexp() / rates[n]) setup = (n = rand(1:num_rates); rates[n] = rand()), OR if I do b2 = @benchmarkable DiffEqJump.update!(rt, 1, rates[1], rates[1]) instead of b2 = @benchmarkable DiffEqJump.update!(rt, n, oldrate, rates[n]) setup = (n = rand(1:num_rates); oldrate = rates[n]; rates[n] = rand()).

Speculations:

  1. It is clear that the issue is related to benchmarking. It follows from observations 1,2 and 4.
  2. It is probable that it is a bug of BenchmarkTools, and not DiffEqJump.
  3. The error seems to be specific for the two structs involved: a priority table and a binary min heap. This is suggested by observation 5.
  4. The error seems to be partially caused by the kwarg setup. This is suggested by observation 7.
ChrisRackauckas commented 3 years ago

nice.

Vilin97 commented 3 years ago

@ChrisRackauckas , sorry for the half-written issue. Ctrl+Enter not giving new line always gets me. Anyway, it's finished now.

Vilin97 commented 3 years ago

I think I found the source of the problem, and it is not a bug. Looks like I was just forgetting to update the priority table with new rates after the previous benchmark modified rates...