buffer of Finch suggestions

[bennn]

Low priority / unpromising ideas that might be worth suggesting to Finch:

• [ ] init_stepper check that 1st arg is positive, non-NaN
• [x] when a mesh contains a NaN, why does Finch run 4x slower and not crash? (TF logging, for sure)
• [x] same thing happens when coefficient returns NaN --- why's there a big slowdown? should it crash?! (turn this into a small example; EDIT: it's a non-issue after all. Plain Finch runs the same with a NaN. TF without logging runs the same too. The slowdown is due to logging https://github.com/utahplt/FloatTracker.jl/issues/19 )
• [ ] ....

[bennn]

The init_stepper suggestion came after I tried injecting 10 times and saw the same result (link) each time:

• inject at call to minimum in finch_interface.jl:1188
• send the NaN as 1st arg to init_stepper
• immediate crash at ceil call time_steppers.jl:27

ERROR: LoadError: InexactError: Int64(NaN)
Stacktrace:
 [1] Int64
   @ ./float.jl:788 [inlined]
 [2] convert(#unused#::Type{Int64}, x::Float64)
   @ Base ./number.jl:7
 [3] setproperty!(x::Stepper, f::Symbol, v::Float64)
   @ Base ./Base.jl:39
 [4] init_stepper(dx::TrackedFloat64, stepper::Stepper)
   @ Finch ~/.julia/dev/Finch/src/time_steppers.jl:27
 [5] macro expansion
   @ ~/.julia/dev/Finch/src/finch_interface.jl:1190 [inlined]
 [6] macro expansion
   @ ~/.julia/packages/TimerOutputs/RsWnF/src/TimerOutput.jl:237 [inlined]
 [7] conservationForm(var::Variable{TrackedFloat64}, cf::String)
   @ Finch ~/.julia/dev/Finch/src/finch_interface.jl:1116
 [8] top-level scope
   @ ~/code/uu/fpx/examples/examples/finch/inj-adv2d-fv.jl:63
in expression starting at /home/ben/code/uu/fpx/examples/examples/finch/inj-adv2d-fv.jl:63

Original program:

``` #= 2D advection using higher order FV or structured or unstructured mesh =# ### If the Finch package has already been added, use this line ######### using Finch # Note: to add the package, first do: ]add "https://github.com/paralab/Finch.git" using Dates using FloatTracker: TrackedFloat64, write_log_to_file, set_inject_nan, set_logger, set_exclude_stacktrace, FunctionRef set_exclude_stacktrace([:prop]) set_logger(filename="inj-adv2d", buffersize=20, cstg=true, cstgArgs=false, cstgLineNum=true) fns = [] ## [FunctionRef(:refel_nodes!, Symbol("refel_nodes.jl"))] libs = [] ##["NBodySimulator", "OrdinaryDiffEq"] now_str = Dates.format(now(), "yyyymmddHHMMss") recording_file = "rand-adv2d-recording_$now_str" println("Recording to $recording_file...") ### If not, use these four lines (working from the examples directory) ### # if !@isdefined(Finch) # include("../Finch.jl"); # using .Finch # end ########################################################################## initFinch("inj-adv2d", TrackedFloat64); # Configuration setup domain(2) solverType(FV) timeStepper(EULER_EXPLICIT,cfl=20000) # NaN-making timestepper println("begin MESH") # a uniform grid of quads on a 0.1 x 0.3 rectangle domain mesh(QUADMESH, elsperdim=[15, 45], bids=4, interval=[0, 0.1, 0, 0.3]) println("end MESH") ######## set_inject_nan(true, 20, 1, fns, libs, record=recording_file) ######## # Variables and BCs u = variable("u", location=CELL) boundary(u, 1, FLUX, "(abs(y-0.06) < 0.033 && sin(3*pi*t)>0) ? 1 : 0") # x=0 boundary(u, 2, NO_BC) # x=0.1 boundary(u, 3, NO_BC) # y=0 boundary(u, 4, NO_BC) # y=0.3 # Time interval and initial condition T = 1.3; # T=10 timeInterval(T) initial(u, "0") # Coefficients coefficient("a", ["0.1*cos(pi*x/2/0.1)","0.3*sin(pi*x/2/0.1)"], type=VECTOR) # advection velocity coefficient("s", ["0.1 * sin(pi*x)^4 * sin(pi*y)^4"]) # source # The "upwind" function applies upwinding to the term (a.n)*u with flow velocity a. # The optional third parameter is for tuning. Default upwind = 0, central = 1. Choose something between these. conservationForm(u, "s + surface(upwind(a,u))"); println("begin SOLVE") solve(u) println("end SOLVE") finalizeFinch() write_log_to_file() ```

[bennn]

Outputs from two poisson1d injections below.

Both injections happen in a similar way:

• when building a grid (grid_from_mesh),
• when inverting a refel_nodes matrix,
• put a NaN inside one cell

Leads to a big slowdown: up from 70s to 300s and from 10GB to 23GB.

Why is it a slowdown and not an error?

Original code, with random injections

``` #= # 1D Poisson, Dirichlet bc # CG, Linear element # Simplest test possible =# using Finch using Dates using FloatTracker: TrackedFloat64, FunctionRef, write_log_to_file, set_inject_nan, set_logger, set_exclude_stacktrace set_exclude_stacktrace([:prop]) set_logger(filename="inj-poisson", buffersize=20, cstg=true, cstgArgs=false, cstgLineNum=true) fns = [] ## [FunctionRef(:refel_nodes!, Symbol("refel_nodes.jl"))] libs = [] ##["NBodySimulator", "OrdinaryDiffEq"] now_str = Dates.format(now(), "yyyymmddHHMMss") recording_file = "rand-poisson-recording_$now_str" println("Recording to $recording_file...") ######## set_inject_nan(true, 20, 1, fns, libs, record=recording_file) ######## initFinch("poisson1d", TrackedFloat64); # Set up the configuration domain(1) # dimension mesh(LINEMESH, elsperdim=180) # build uniform LINEMESH with 180 elements u = variable("u") # make a scalar variable with symbol u testSymbol("v") # sets the symbol for a test function boundary(u, 1, DIRICHLET, 0) # boundary condition for BID 1 is Dirichlet with value 0 # Write the weak form coefficient("f", "-100*pi*pi*sin(10*pi*x)*sin(pi*x) - pi*pi*sin(10*pi*x)*sin(pi*x) + 20*pi*pi*cos(10*pi*x)*cos(pi*x)") weakForm(u, "-grad(u)*grad(v) - f*v") # exportCode("poisson1dcode"); # importCode("poisson1dcode"); solve(u); finalizeFinch() write_log_to_file() ```

1

Recording file:

21, lu.jl

Injection site + stacktrace, final output:

INJECT - (1.224744871391589, -1.224744871391589)...
run_or_inject at /home/ben/.julia/dev/FloatTracker/src/TrackedFloat.jl:39
- at /home/ben/.julia/dev/FloatTracker/src/TrackedFloat.jl:105
#generic_lufact!#178 at /home/ben/code/julia/julia-1.8.0/usr/share/julia/stdlib/v1.8/LinearAlgebra/src/lu.jl:181
generic_lufact!##kw at /home/ben/code/julia/julia-1.8.0/usr/share/julia/stdlib/v1.8/LinearAlgebra/src/lu.jl:137
#lu!#177 at /home/ben/code/julia/julia-1.8.0/usr/share/julia/stdlib/v1.8/LinearAlgebra/src/lu.jl:135
lu!##kw at /home/ben/code/julia/julia-1.8.0/usr/share/julia/stdlib/v1.8/LinearAlgebra/src/lu.jl:135
#lu#179 at /home/ben/code/julia/julia-1.8.0/usr/share/julia/stdlib/v1.8/LinearAlgebra/src/lu.jl:279
lu at /home/ben/code/julia/julia-1.8.0/usr/share/julia/stdlib/v1.8/LinearAlgebra/src/lu.jl:278
lu at /home/ben/code/julia/julia-1.8.0/usr/share/julia/stdlib/v1.8/LinearAlgebra/src/lu.jl:278
inv at /home/ben/code/julia/julia-1.8.0/usr/share/julia/stdlib/v1.8/LinearAlgebra/src/dense.jl:893
build_refel at /home/ben/.julia/dev/Finch/src/refel.jl:257
#grid_from_mesh#21 at /home/ben/.julia/dev/Finch/src/grid.jl:159
grid_from_mesh##kw at /home/ben/.julia/dev/Finch/src/grid.jl:80
macro expansion at /home/ben/.julia/dev/Finch/src/Finch.jl:242
macro expansion at /home/ben/.julia/packages/TimerOutputs/RsWnF/src/TimerOutput.jl:237
#add_mesh#134 at /home/ben/.julia/dev/Finch/src/Finch.jl:145
add_mesh##kw at /home/ben/.julia/dev/Finch/src/Finch.jl:142
macro expansion at /home/ben/.julia/dev/Finch/src/finch_interface.jl:414
macro expansion at /home/ben/.julia/packages/TimerOutputs/RsWnF/src/TimerOutput.jl:237
#mesh#81 at /home/ben/.julia/dev/Finch/src/finch_interface.jl:359
mesh##kw at /home/ben/.julia/dev/Finch/src/finch_interface.jl:357
top-level scope at /home/ben/code/uu/fpx/examples/examples/finch/inj-poisson.jl:28
eval at ./boot.jl:368
include_string at ./loading.jl:1428
_include at ./loading.jl:1488
include at ./Base.jl:419
exec_options at ./client.jl:303
_start at ./client.jl:522

 ──────────────────────────────────────────────────────────────────────────
                                  Time                    Allocations      
                         ───────────────────────   ────────────────────────
    Tot / % measured:          300s /  88.6%           23.5GiB /  71.4%    

 Section         ncalls     time    %tot     avg     alloc    %tot      avg
 ──────────────────────────────────────────────────────────────────────────
 Solve                1     239s   89.8%    239s   14.9GiB   88.7%  14.9GiB
   FE_solve           1     238s   89.7%    238s   14.9GiB   88.5%  14.9GiB
     lin_solve        1     228s   85.8%    228s   13.2GiB   78.5%  13.2GiB
     assembly         1   78.6ms    0.0%  78.6ms   39.0MiB    0.2%  39.0MiB
     scatter          1   1.44μs    0.0%  1.44μs     0.00B    0.0%    0.00B
 Mesh                 1    17.8s    6.7%   17.8s   1.34GiB    8.0%  1.34GiB
   mesh2grid          1    11.9s    4.5%   11.9s    811MiB    4.7%   811MiB
   gen/read           1    142ms    0.1%   142ms   6.75MiB    0.0%  6.75MiB
   geo factors        1   85.4ms    0.0%  85.4ms   14.9MiB    0.1%  14.9MiB
 CodeGen              1    9.35s    3.5%   9.35s    579MiB    3.4%   579MiB
 ──────────────────────────────────────────────────────────────────────────
Finch has completed.

2

Recording File:

24, triangular.jl

Injection site + stacktrace, final output:

INJECT - (0.0, 1.0)...
run_or_inject at /home/ben/.julia/dev/FloatTracker/src/TrackedFloat.jl:39
- at /home/ben/.julia/dev/FloatTracker/src/TrackedFloat.jl:105
ldiv! at /home/ben/code/julia/julia-1.8.0/usr/share/julia/stdlib/v1.8/LinearAlgebra/src/triangular.jl:1245
ldiv! at /home/ben/code/julia/julia-1.8.0/usr/share/julia/stdlib/v1.8/LinearAlgebra/src/triangular.jl:1258
ldiv! at /home/ben/code/julia/julia-1.8.0/usr/share/julia/stdlib/v1.8/LinearAlgebra/src/lu.jl:409
ldiv! at /home/ben/code/julia/julia-1.8.0/usr/share/julia/stdlib/v1.8/LinearAlgebra/src/factorization.jl:126
inv! at /home/ben/code/julia/julia-1.8.0/usr/share/julia/stdlib/v1.8/LinearAlgebra/src/lu.jl:490
inv at /home/ben/code/julia/julia-1.8.0/usr/share/julia/stdlib/v1.8/LinearAlgebra/src/dense.jl:893
build_refel at /home/ben/.julia/dev/Finch/src/refel.jl:257
#grid_from_mesh#21 at /home/ben/.julia/dev/Finch/src/grid.jl:159
grid_from_mesh##kw at /home/ben/.julia/dev/Finch/src/grid.jl:80
macro expansion at /home/ben/.julia/dev/Finch/src/Finch.jl:242
macro expansion at /home/ben/.julia/packages/TimerOutputs/RsWnF/src/TimerOutput.jl:237
#add_mesh#134 at /home/ben/.julia/dev/Finch/src/Finch.jl:145
add_mesh##kw at /home/ben/.julia/dev/Finch/src/Finch.jl:142
macro expansion at /home/ben/.julia/dev/Finch/src/finch_interface.jl:414
macro expansion at /home/ben/.julia/packages/TimerOutputs/RsWnF/src/TimerOutput.jl:237
#mesh#81 at /home/ben/.julia/dev/Finch/src/finch_interface.jl:359
mesh##kw at /home/ben/.julia/dev/Finch/src/finch_interface.jl:357
top-level scope at /home/ben/code/uu/fpx/examples/examples/finch/inj-poisson.jl:28
eval at ./boot.jl:368
include_string at ./loading.jl:1428
_include at ./loading.jl:1488
include at ./Base.jl:419
exec_options at ./client.jl:303
_start at ./client.jl:522

 ──────────────────────────────────────────────────────────────────────────
                                  Time                    Allocations      
                         ───────────────────────   ────────────────────────
    Tot / % measured:          300s /  88.6%           23.5GiB /  71.4%    

 Section         ncalls     time    %tot     avg     alloc    %tot      avg
 ──────────────────────────────────────────────────────────────────────────
 Solve                1     239s   89.9%    239s   14.9GiB   88.7%  14.9GiB
   FE_solve           1     238s   89.8%    238s   14.9GiB   88.5%  14.9GiB
     lin_solve        1     229s   86.2%    229s   13.2GiB   78.6%  13.2GiB
     assembly         1   76.6ms    0.0%  76.6ms   38.0MiB    0.2%  38.0MiB
     scatter          1   1.50μs    0.0%  1.50μs     0.00B    0.0%    0.00B
 Mesh                 1    17.6s    6.6%   17.6s   1.34GiB    8.0%  1.34GiB
   mesh2grid          1    11.7s    4.4%   11.7s    808MiB    4.7%   808MiB
   gen/read           1    142ms    0.1%   142ms   6.75MiB    0.0%  6.75MiB
   geo factors        1   83.6ms    0.0%  83.6ms   13.3MiB    0.1%  13.3MiB
 CodeGen              1    9.10s    3.4%   9.10s    579MiB    3.4%   579MiB
 ──────────────────────────────────────────────────────────────────────────
Finch has completed.