search

mabarnes / moment_kinetics

Other
2 stars 4 forks source link

Optimize parallelisation of Fokker Planck collision operator #176

Closed johnomotani closed 3 months ago

johnomotani commented 6 months ago

Introduces new shared-memory parallelisation type, started by begin_s_r_z_anyv_region(), intended for use with the collision operator. See new docs here: https://mabarnes.github.io/moment_kinetics/previews/PR176/developing/#Collision-operator-and-anyv-region.

Also adds some optimizations to speed up serial performance of the assemble_explicit_collision_operator_rhs_parallel!() function.

Still draft because I am running some tests to check that the optimizations actually speed things up. An initial test showed some odd results for the parallelisation upgrade, which looked like decreased performance when only using distributed-MPI parallelisation. I'm hoping it's just an anomaly because the node some tests ran on was going slowly for no particular reason, so need to re-run the tests.

johnomotani commented 4 months ago

I think this PR is ready to merge now. I haven't touched it since January 17th, and I'd benchmarked that it speeds up the collision operator in serial and also improves parallel scaling.

Performance test on ARCHER2, based on input in examples/fokker-planck-1D2V/fokker-planck-1D2V-even_nz-shorttest-nstep200.toml: the code before this PR was most efficient when using distributed MPI parallelism as much as possible (so setting z_nelement_local=1). cores time (s)
16 74.2
32 43.2
64 34.6
128 26.3
256 29.9
The code from this PR is efficient for pretty much any split between distributed- and shared-memory parallelism, so here are results with different values of z_nelement_local (times again in seconds). cores z_nelement_local = 1 z_nelement_local = 2 z_nelement_local = 4 z_nelement_local = 8 z_nelement_local = 16
1 469
16 49.0 47.9 46.3 47.5 45.2
32 28.6 26.4 27.5 27.3 27.8
64 18.9 20.2 18.2 21.1 21.3
128 11.0 12.4 13.6 14.0 16.6
256 7.49 7.89 6.92 8.54
mrhardman commented 4 months ago

Sounds good @johnomotani! Thank you for doing these tests. I would like to take a look through the optimised code to just make sure that I can understand what it is doing, or ask for clarification. Hopefully this only takes a few days.

Would you be able to try running the scripts https://github.com/mabarnes/moment_kinetics/blob/optimize-fokker-planck/test_scripts/2D_FEM_assembly_test.jl and https://github.com/mabarnes/moment_kinetics/blob/optimize-fokker-planck/test_scripts/fkpl_direct_integration_test.jl to reproduce the scaling in my report for init time and run time versus problem size?

johnomotani commented 3 months ago

Would you be able to try running the scripts https://github.com/mabarnes/moment_kinetics/blob/optimize-fokker-planck/test_scripts/2D_FEM_assembly_test.jl and https://github.com/mabarnes/moment_kinetics/blob/optimize-fokker-planck/test_scripts/fkpl_direct_integration_test.jl to reproduce the scaling in my report for init time and run time versus problem size?

Those scripts now run. Here are the timing plots:

To get timings directly comparable to your report, you'd have to run them on the same machine.

Edit: for completeness, the command line output I got from the scripts was

$ mpirun -np 2 bin/julia --project -O3 --check-bounds=no test_scripts/2D_FEM_assembly_test.jl 
  Activating  Activating project at `~/physics/moment_kinetics-master`
 project at `~/physics/moment_kinetics-master`
made inputs
vpa: ngrid: 5 nelement: 16 Lvpa: 12.0
vperp: ngrid: 5 nelement: 8 Lvperp: 6.0
made inputs
vpa: ngrid: 5 nelement: 16 Lvpa: 12.0
vperp: ngrid: 5 nelement: 8 Lvperp: 6.0
setting up GL quadrature   15:03:40
beginning (boundary) weights calculation   15:03:40
finished (boundary) weights calculation   15:03:42
begin elliptic operator assignment   15:03:42
finished elliptic operator constructor assignment   15:03:42
finished LU decomposition initialisation   15:03:42
finished YY array calculation   15:03:42
maximum(d2fvpavperp_dvpa2_err): 0.001530306391981484
maximum(d2fvpavperp_dvperp2_err): 0.19354033782475888
begin C calculation   15:03:42
finished C calculation   15:03:45
H boundary data:
max(lower_vpa_err) = 2.0248975496350852e-8
max(upper_vpa_err) = 2.927859837908997e-8
max(upper_vperp_err) = 2.2062522864185752e-8
dHdvpa boundary data:
max(lower_vpa_err) = 3.102063106386055e-9
max(upper_vpa_err) = 6.563723030372781e-9
max(upper_vperp_err) = 1.364009852983239e-9
dHdvperp boundary data:
max(lower_vpa_err) = 7.291533935319117e-8
max(upper_vpa_err) = 1.4561553543686323e-7
max(upper_vperp_err) = 2.4891693521308933e-7
G boundary data:
max(lower_vpa_err) = 1.2397879185499505e-6
max(upper_vpa_err) = 1.0480128835865798e-6
max(upper_vperp_err) = 1.2397879185499505e-6
dGdvperp boundary data:
max(lower_vpa_err) = 5.9302478292444505e-6
max(upper_vpa_err) = 6.9899921779637e-6
max(upper_vperp_err) = 9.03211198888254e-6
d2Gdvperp2 boundary data:
max(lower_vpa_err) = 1.308518969567185e-6
max(upper_vpa_err) = 1.840856089951881e-6
max(upper_vperp_err) = 5.662285593455163e-7
d2Gdvperpdvpa boundary data:
max(lower_vpa_err) = 4.998436504283221e-7
max(upper_vpa_err) = 6.978403086776996e-7
max(upper_vperp_err) = 5.726848005538354e-7
d2Gdvpa2 boundary data:
max(lower_vpa_err) = 6.482958898879065e-9
max(upper_vpa_err) = 1.071649759210036e-8
max(upper_vperp_err) = 2.183564176694297e-8
maximum(H_M_err): 3.2307878661264056e-6 L2(H_M_err): 1.4759478084158561e-7
maximum(dHdvpa_M_err): 2.6877539530012484e-5 L2(dHdvpa_M_err): 8.963500670324541e-7
maximum(dHdvperp_M_err): 7.788624189868087e-5 L2(dHdvperp_M_err): 1.4331325405348176e-6
maximum(G_M_err): 1.2397879185499505e-6 L2(G_M_err): 7.386176435456055e-7
maximum(d2Gdvpa2_M_err): 4.178327956849692e-6 L2(d2Gdvpa2_M_err): 1.821946351843197e-7
maximum(dGdvperp_M_err): 2.682324361623356e-5 L2(dGdvperp_M_err): 5.722789550834238e-6
maximum(d2Gdvperpdvpa_M_err): 1.4771340498211452e-5 L2(d2Gdvperpdvpa_M_err): 5.171597902142012e-7
maximum(d2Gdvperp2_M_err): 2.7551494087285633e-5 L2(d2Gdvperp2_M_err): 6.002712904357497e-7
maximum(C_M_err): 0.010526301815059582 L2(C_M_err): 0.0001763571352607852
dSdt: 9.701390531776414e-7 should be >0.0
delta_n: -5.681038351680817e-15
delta_upar: 1.4330830109698342e-8
delta_pressure: 5.016552856231658e-7
fkpl_C_G_H_max_test_ngrid_5_GLL.pdf
[[0.010526301815059582], [3.2307878661264056e-6], [1.2397879185499505e-6], [0.000244140625], [3.814697265625e-6]]
fkpl_coeffs_max_test_ngrid_5_GLL.pdf
[[2.6877539530012484e-5], [7.788624189868087e-5], [2.7551494087285633e-5], [4.178327956849692e-6], [1.4771340498211452e-5], [2.682324361623356e-5], [0.000244140625], [3.814697265625e-6]]
fkpl_C_G_H_L2_test_ngrid_5_GLL.pdf
[[0.0001763571352607852], [1.4759478084158561e-7], [7.386176435456055e-7], [0.000244140625], [3.814697265625e-6]]
fkpl_coeffs_L2_test_ngrid_5_GLL.pdf
[[8.963500670324541e-7], [1.4331325405348176e-6], [6.002712904357497e-7], [1.821946351843197e-7], [5.171597902142012e-7], [5.722789550834238e-6], [0.000244140625], [3.814697265625e-6]]
fkpl_conservation_test_ngrid_5_GLL.pdf
[[0.010526301815059582], [0.0001763571352607852], [5.681038351680817e-15], [1.4330830109698342e-8], [5.016552856231658e-7], [0.000244140625], [3.814697265625e-6]]
fkpl_timing_test_ngrid_5_GLL.pdf
[[2609.0], [10273.0], [64.0], [512.0]]
made inputs
vpa: ngrid: 3 nelement: 16 Lvpa: 12.0
vperp: ngrid: 3 nelement: 8 Lvperp: 6.0
made inputs
vpa: ngrid: 3 nelement: 16 Lvpa: 12.0
vperp: ngrid: 3 nelement: 8 Lvperp: 6.0
setting up GL quadrature   15:03:49
beginning (boundary) weights calculation   15:03:49
finished (boundary) weights calculation   15:03:49
begin elliptic operator assignment   15:03:49
finished elliptic operator constructor assignment   15:03:49
finished LU decomposition initialisation   15:03:49
finished YY array calculation   15:03:49
maximum(d2fvpavperp_dvpa2_err): 0.07114121456450606
maximum(d2fvpavperp_dvperp2_err): 1.432583187758945
begin C calculation   15:03:49
made inputs
finished C calculation   15:03:49
H boundary data:
max(lower_vpa_err) = 0.00010521317050291024
max(upper_vpa_err) = 0.0001434634556239378
max(upper_vperp_err) = 0.00013367040020351162vpa: ngrid: 
dHdvpa boundary data:
max(lower_vpa_err) = 1.391794144315131e-5
3 nelement: 32 Lvpa: max(upper_vpa_err) = 12.0
2.614313446594463e-5
vperp: ngrid: max(upper_vperp_err) = 3 nelement: 9.205486475835287e-6
16 Lvperp: dHdvperp boundary data:
6.0
max(lower_vpa_err) = 2.5827336280091415e-6
max(upper_vpa_err) = 4.3826479867812435e-6
max(upper_vperp_err) = 1.4666550697914338e-5
G boundary data:
max(lower_vpa_err) = 0.007309838493426923
max(upper_vpa_err) = 0.006243127285856076
max(upper_vperp_err) = 0.007309838493426923
dGdvperp boundary data:
max(lower_vpa_err) = 0.00024447470374355795
max(upper_vpa_err) = 0.0002833068173460074
max(upper_vperp_err) = 0.0003769637590347319
d2Gdvperp2 boundary data:
max(lower_vpa_err) = 4.6897506094129016e-5
max(upper_vpa_err) = 5.343305220187333e-5
max(upper_vperp_err) = 2.9596110035445822e-5
d2Gdvperpdvpa boundary data:
max(lower_vpa_err) = 1.761348162565568e-5
max(upper_vpa_err) = 2.1252799052927374e-5
max(upper_vperp_err) = 2.3755392333506264e-5
d2Gdvpa2 boundary data:
max(lower_vpa_err) = 3.4762432362590856e-5
max(upper_vpa_err) = 5.5404461270128214e-5
max(upper_vperp_err) = 0.00013259739930668313
maximum(H_M_err): 0.0013417904432995176 L2(H_M_err): 0.0001747721983459617
maximum(dHdvpa_M_err): 0.004350898874976694 L2(dHdvpa_M_err): 0.000162645849054975
maximum(dHdvperp_M_err): 0.0037118516663812856 L2(dHdvperp_M_err): 0.0002401170164049581
maximum(G_M_err): 0.007309838493426923 L2(G_M_err): 0.004607127392495961
maximum(d2Gdvpa2_M_err): 0.0011108514806762448 L2(d2Gdvpa2_M_err): 0.00013357704009381364
maximum(dGdvperp_M_err): 0.002180979207375186 L2(dGdvperp_M_err): 0.0003250746046973753
maximum(d2Gdvperpdvpa_M_err): 0.0013372090347524984 L2(d2Gdvperpdvpa_M_err): 9.803183492853563e-5
maximum(d2Gdvperp2_M_err): 0.0025562055668788553 L2(d2Gdvperp2_M_err): 9.845888859211535e-5
maximum(C_M_err): 0.14664181770220974 L2(C_M_err): 0.003973958788863551
dSdt: 0.0015151029649126298 should be >0.0
delta_n: -4.166346993219358e-15
delta_upar: -1.9728084934773465e-6
delta_pressure: 0.0005673588325043538
made inputs
vpa: ngrid: 3 nelement: 32 Lvpa: 12.0
vperp: ngrid: 3 nelement: 16 Lvperp: 6.0
setting up GL quadrature   15:03:49
beginning (boundary) weights calculation   15:03:49
finished (boundary) weights calculation   15:03:50
begin elliptic operator assignment   15:03:50
finished elliptic operator constructor assignment   15:03:50
finished LU decomposition initialisation   15:03:50
finished YY array calculation   15:03:50
maximum(d2fvpavperp_dvpa2_err): 0.01071077877726001
maximum(d2fvpavperp_dvperp2_err): 0.5135608222052297
begin C calculation   15:03:51
made inputsfinished C calculation   15:03:51
H boundary data:
max(lower_vpa_err) = 5.692079352831447e-6
max(upper_vpa_err) = 8.286989191558236e-6
max(upper_vperp_err) = 5.842997205296463e-6

dHdvpa boundary data:
max(lower_vpa_err) = 8.980283882110973e-7vpa: ngrid: 3
max(upper_vpa_err) = 1.8588453679357908e-6 nelement: 64
max(upper_vperp_err) = 3.635418752041819e-7
dHdvperp boundary data: Lvpa: 12.0
vperp: ngrid: 3 nelement: 32
max(lower_vpa_err) =  Lvperp: 6.01.443065743133437e-7
max(upper_vpa_err) = 3.6319379611605973e-7

max(upper_vperp_err) = 2.414672679029728e-7
G boundary data:
max(lower_vpa_err) = 0.00033407447249622635
max(upper_vpa_err) = 0.000281037265042805
max(upper_vperp_err) = 0.00033407447249622635
dGdvperp boundary data:
max(lower_vpa_err) = 8.675416379944068e-6
max(upper_vpa_err) = 1.0495073440974778e-5
max(upper_vperp_err) = 1.2993610074896189e-5
d2Gdvperp2 boundary data:
max(lower_vpa_err) = 2.248665537218919e-6
max(upper_vpa_err) = 3.6901255116850162e-6
max(upper_vperp_err) = 6.109648304410825e-7
d2Gdvperpdvpa boundary data:
max(lower_vpa_err) = 1.021284195996952e-6
max(upper_vpa_err) = 1.9041938305874861e-6
max(upper_vperp_err) = 9.466084754938819e-7
d2Gdvpa2 boundary data:
max(lower_vpa_err) = 1.8053402811579744e-6
max(upper_vpa_err) = 3.044281904701629e-6
max(upper_vperp_err) = 5.717625302575913e-6
maximum(H_M_err): 0.00010190387803010736 L2(H_M_err): 1.0560604724110431e-5
maximum(dHdvpa_M_err): 0.0002460686458479744 L2(dHdvpa_M_err): 1.0465599140049831e-5
maximum(dHdvperp_M_err): 0.001137436050168611 L2(dHdvperp_M_err): 1.655450651195722e-5
maximum(G_M_err): 0.00033407447249622635 L2(G_M_err): 0.0001938291088798128
maximum(d2Gdvpa2_M_err): 6.0456633486194455e-5 L2(d2Gdvpa2_M_err): 7.463732920291641e-6
maximum(dGdvperp_M_err): 0.0004204533409555497 L2(dGdvperp_M_err): 1.912016983346137e-5
maximum(d2Gdvperpdvpa_M_err): 0.00020521224594612607 L2(d2Gdvperpdvpa_M_err): 6.277136810119457e-6
maximum(d2Gdvperp2_M_err): 0.00010505104732272397 L2(d2Gdvperp2_M_err): 7.774320987060786e-6
maximum(C_M_err): 0.06546584218520007 L2(C_M_err): 0.0013151792082636148
dSdt: 0.00021795100315665457 should be >0.0
delta_n: -5.036600558624537e-15
delta_upar: 3.1975568273648396e-8
delta_pressure: 0.00024192437846768007
made inputs
vpa: ngrid: 3 nelement: 64 Lvpa: 12.0
vperp: ngrid: 3 nelement: 32 Lvperp: 6.0
setting up GL quadrature   15:03:51
beginning (boundary) weights calculation   15:03:51
finished (boundary) weights calculation   15:04:01
begin elliptic operator assignment   15:04:01
finished elliptic operator constructor assignment   15:04:01
finished LU decomposition initialisation   15:04:02
finished YY array calculation   15:04:02
maximum(d2fvpavperp_dvpa2_err): 0.006738038244671785
maximum(d2fvpavperp_dvperp2_err): 0.14829571258344876
begin C calculation   15:04:02
finished C calculation   15:04:02
H boundary data:
max(lower_vpa_err) = 6.523535629188171e-7
max(upper_vpa_err) = 9.373247351063263e-7
max(upper_vperp_err) = 7.057360648377031e-7
dHdvpa boundary data:
max(lower_vpa_err) = 9.933104548956462e-8
max(upper_vpa_err) = 2.0324850254910265e-7
max(upper_vperp_err) = 4.399804100739535e-8
dHdvperp boundary data:
max(lower_vpa_err) = 3.27409227654854e-8made inputs
max(upper_vpa_err) = 7.029340848459242e-8
max(upper_vperp_err) = 8.913665790200032e-8

G boundary data:
max(lower_vpa_err) = 3.986574765413309e-5
vpa: ngrid: 3 nelement: max(upper_vpa_err) = 3.3669028512761656e-5128
max(upper_vperp_err) =  Lvpa: 3.986574765413309e-5
12.0
vperp: ngrid: dGdvperp boundary data:
max(lower_vpa_err) = 3 nelement: 642.4932005541211666e-6
max(upper_vpa_err) =  Lvperp: 6.02.955639210822092e-6
max(upper_vperp_err) = 
3.784415427920962e-6
d2Gdvperp2 boundary data:
max(lower_vpa_err) = 5.71008685001928e-7
max(upper_vpa_err) = 8.363745355455254e-7
max(upper_vperp_err) = 2.2021083746770476e-7
d2Gdvperpdvpa boundary data:
max(lower_vpa_err) = 2.2573767368555542e-7
max(upper_vpa_err) = 3.448664878391572e-7
max(upper_vperp_err) = 2.463557735657429e-7
d2Gdvpa2 boundary data:
max(lower_vpa_err) = 2.088053318652361e-7
max(upper_vpa_err) = 3.474603478964555e-7
max(upper_vperp_err) = 6.934792424750924e-7
maximum(H_M_err): 1.34865876035839e-5 L2(H_M_err): 1.1211133966439615e-6
maximum(dHdvpa_M_err): 1.8428574157458577e-5 L2(dHdvpa_M_err): 6.729063532024829e-7
maximum(dHdvperp_M_err): 0.00017708444628909692 L2(dHdvperp_M_err): 1.4475601598871342e-6
maximum(G_M_err): 3.986574765413309e-5 L2(G_M_err): 2.3619023452776415e-5
maximum(d2Gdvpa2_M_err): 5.637868173202776e-6 L2(d2Gdvpa2_M_err): 7.771311681049361e-7
maximum(dGdvperp_M_err): 6.061758248512503e-5 L2(dGdvperp_M_err): 3.308912953962959e-6
maximum(d2Gdvperpdvpa_M_err): 3.0212422638684414e-5 L2(d2Gdvperpdvpa_M_err): 5.515581454650211e-7
maximum(d2Gdvperp2_M_err): 9.858040189603123e-6 L2(d2Gdvperp2_M_err): 6.553611260568663e-7
maximum(C_M_err): 0.05541458710231074 L2(C_M_err): 0.0007241001038272485
dSdt: 3.447808357980628e-5 should be >0.0
delta_n: -1.290253196769475e-14
delta_upar: 1.831809922562343e-10
delta_pressure: 2.4223831206815676e-5
made inputs
vpa: ngrid: 3 nelement: 128 Lvpa: 12.0
vperp: ngrid: 3 nelement: 64 Lvperp: 6.0
setting up GL quadrature   15:04:02
beginning (boundary) weights calculation   15:04:02
finished (boundary) weights calculation   15:05:19
begin elliptic operator assignment   15:05:19
finished elliptic operator constructor assignment   15:05:20
finished LU decomposition initialisation   15:05:23
finished YY array calculation   15:05:23
maximum(d2fvpavperp_dvpa2_err): 0.002055685203458557
maximum(d2fvpavperp_dvperp2_err): 0.03882083964807048
begin C calculation   15:05:24
finished C calculation   15:05:24
H boundary data:
max(lower_vpa_err) = 4.5880787363783426e-8
max(upper_vpa_err) = 6.581257183668754e-8
max(upper_vperp_err) = 4.99656731267617e-8
dHdvpa boundary data:
max(lower_vpa_err) = 6.954421916810505e-9
max(upper_vpa_err) = 1.4199531953029698e-8
max(upper_vperp_err) = 3.1189005180592977e-9
dHdvperp boundary data:
max(lower_vpa_err) = 2.5770653267614962e-9
max(upper_vpa_err) = 5.456208868395707e-9
max(upper_vperp_err) = 7.380039239446301e-9
G boundary data:
max(lower_vpa_err) = 2.81819214187351e-6
max(upper_vpa_err) = 2.3813029121910745e-6
max(upper_vperp_err) = 2.81819214187351e-6
dGdvperp boundary data:
max(lower_vpa_err) = 1.9904590409236533e-7
max(upper_vpa_err) = 2.3567749052322284e-7
max(upper_vperp_err) = 3.023670857027483e-7
d2Gdvperp2 boundary data:
max(lower_vpa_err) = 4.527422850508778e-8
max(upper_vpa_err) = 6.575671626651847e-8
max(upper_vperp_err) = 1.788273376601701e-8
d2Gdvperpdvpa boundary data:
max(lower_vpa_err) = 1.774561694811716e-8
max(upper_vpa_err) = 2.6621849669172803e-8
max(upper_vperp_err) = 1.9569094060478687e-8
d2Gdvpa2 boundary data:
max(lower_vpa_err) = 1.4696307559791677e-8
max(upper_vpa_err) = 2.4423288946806565e-8
max(upper_vperp_err) = 4.911970288845602e-8
made inputs
vpa: ngrid: 3 nelement: 256 Lvpa: 12.0
vperp: ngrid: 3 nelement: 128 Lvperp: 6.0
maximum(H_M_err): 1.3267648804049514e-6 L2(H_M_err): 7.81262205501613e-8
maximum(dHdvpa_M_err): 1.4244949509967242e-6 L2(dHdvpa_M_err): 4.2404287911779546e-8
maximum(dHdvperp_M_err): 2.363081817918604e-5 L2(dHdvperp_M_err): 1.1724960326123184e-7
maximum(G_M_err): 2.81819214187351e-6 L2(G_M_err): 1.674074568243166e-6
maximum(d2Gdvpa2_M_err): 4.824409535197915e-7 L2(d2Gdvpa2_M_err): 5.4139239706282395e-8
maximum(dGdvperp_M_err): 7.936752848375708e-6 L2(dGdvperp_M_err): 2.5415135641422175e-7
maximum(d2Gdvperpdvpa_M_err): 3.978565377478309e-6 L2(d2Gdvperpdvpa_M_err): 4.0530125435562106e-8
maximum(d2Gdvperp2_M_err): 9.612873482778284e-7 L2(d2Gdvperp2_M_err): 4.426571003517553e-8
maximum(C_M_err): 0.01974555428819708 L2(C_M_err): 0.0002301430438426956
dSdt: 2.7821538513621585e-6 should be >0.0
delta_n: -4.0735789245218825e-14
delta_upar: -4.4272339207844243e-11
delta_pressure: 1.7384082273717221e-6
made inputs
vpa: ngrid: 3 nelement: 256 Lvpa: 12.0
vperp: ngrid: 3 nelement: 128 Lvperp: 6.0
setting up GL quadrature   15:05:24
beginning (boundary) weights calculation   15:05:24
finished (boundary) weights calculation   15:14:15
begin elliptic operator assignment   15:14:15
finished elliptic operator constructor assignment   15:14:17
finished LU decomposition initialisation   15:14:28
finished YY array calculation   15:14:28
maximum(d2fvpavperp_dvpa2_err): 0.0005402207734037656
maximum(d2fvpavperp_dvperp2_err): 0.009827558870158182
begin C calculation   15:14:28
finished C calculation   15:14:30
H boundary data:
max(lower_vpa_err) = 2.9492218944771054e-9
max(upper_vpa_err) = 4.227846961413206e-9
max(upper_vperp_err) = 3.2164381991162116e-9
dHdvpa boundary data:
max(lower_vpa_err) = 4.4719353220479263e-10
max(upper_vpa_err) = 8.998579728580758e-10
max(upper_vperp_err) = 2.014591427706991e-10
dHdvperp boundary data:
max(lower_vpa_err) = 1.7010619128565718e-10
max(upper_vpa_err) = 3.590513424772679e-10
max(upper_vperp_err) = 4.920191545854102e-10
G boundary data:
max(lower_vpa_err) = 1.8135052393120077e-7
max(upper_vpa_err) = 1.5325740498894902e-7
max(upper_vperp_err) = 1.8135052393120077e-7
dGdvperp boundary data:
max(lower_vpa_err) = 1.3175653790220565e-8
max(upper_vpa_err) = 1.5596308799992187e-8
max(upper_vperp_err) = 2.0017995705501335e-8
d2Gdvperp2 boundary data:
max(lower_vpa_err) = 3.1404822353753303e-9
max(upper_vpa_err) = 4.395971781034902e-9
max(upper_vperp_err) = 1.1877107908286888e-9
d2Gdvperpdvpa boundary data:
max(lower_vpa_err) = 1.1736138103013438e-9
max(upper_vpa_err) = 1.7432814497508886e-9
max(upper_vperp_err) = 1.2978935912122402e-9
d2Gdvpa2 boundary data:
max(lower_vpa_err) = 9.38394167926404e-10
max(upper_vpa_err) = 1.5696534627629433e-9
max(upper_vperp_err) = 3.157710648515888e-9
maximum(H_M_err): 1.0991796206560878e-7 L2(H_M_err): 5.010904657519494e-9
maximum(dHdvpa_M_err): 1.0727694166323687e-7 L2(dHdvpa_M_err): 2.6540328192269194e-9
maximum(dHdvperp_M_err): 3.012764448863739e-6 L2(dHdvperp_M_err): 8.70653816670338e-9
maximum(G_M_err): 1.8135052393120077e-7 L2(G_M_err): 1.0779133252210256e-7
maximum(d2Gdvpa2_M_err): 3.805709614113084e-8 L2(d2Gdvpa2_M_err): 3.4697846705639162e-9
maximum(dGdvperp_M_err): 1.0064603011717549e-6 L2(dGdvperp_M_err): 1.6755593824504363e-8
maximum(d2Gdvperpdvpa_M_err): 5.058584911594912e-7 L2(d2Gdvperpdvpa_M_err): 2.777002863641979e-9
maximum(d2Gdvperp2_M_err): 8.18380293532428e-8 L2(d2Gdvperp2_M_err): 2.818766792141885e-9
maximum(C_M_err): 0.005687615069521508 L2(C_M_err): 6.145613936354202e-5
dSdt: 1.8605910763709205e-7 should be >0.0
delta_n: -1.5966001363131454e-13
delta_upar: 1.447120980544581e-12
delta_pressure: 1.1287451524574988e-7
fkpl_C_G_H_max_test_ngrid_3_GLL.pdf
[[0.14664181770220974, 0.06546584218520007, 0.05541458710231074, 0.01974555428819708, 0.005687615069521508], [0.0013417904432995176, 0.00010190387803010736, 1.34865876035839e-5, 1.3267648804049514e-6, 1.0991796206560878e-7], [0.007309838493426923, 0.00033407447249622635, 3.986574765413309e-5, 2.81819214187351e-6, 1.8135052393120077e-7], [0.015625, 0.00390625, 0.0009765625, 0.000244140625, 6.103515625e-5], [0.000244140625, 1.52587890625e-5, 9.5367431640625e-7, 5.960464477539063e-8, 3.725290298461914e-9]]
fkpl_coeffs_max_test_ngrid_3_GLL.pdf
[[0.004350898874976694, 0.0002460686458479744, 1.8428574157458577e-5, 1.4244949509967242e-6, 1.0727694166323687e-7], [0.0037118516663812856, 0.001137436050168611, 0.00017708444628909692, 2.363081817918604e-5, 3.012764448863739e-6], [0.0025562055668788553, 0.00010505104732272397, 9.858040189603123e-6, 9.612873482778284e-7, 8.18380293532428e-8], [0.0011108514806762448, 6.0456633486194455e-5, 5.637868173202776e-6, 4.824409535197915e-7, 3.805709614113084e-8], [0.0013372090347524984, 0.00020521224594612607, 3.0212422638684414e-5, 3.978565377478309e-6, 5.058584911594912e-7], [0.002180979207375186, 0.0004204533409555497, 6.061758248512503e-5, 7.936752848375708e-6, 1.0064603011717549e-6], [0.015625, 0.00390625, 0.0009765625, 0.000244140625, 6.103515625e-5], [0.000244140625, 1.52587890625e-5, 9.5367431640625e-7, 5.960464477539063e-8, 3.725290298461914e-9]]
fkpl_C_G_H_L2_test_ngrid_3_GLL.pdf
[[0.003973958788863551, 0.0013151792082636148, 0.0007241001038272485, 0.0002301430438426956, 6.145613936354202e-5], [0.0001747721983459617, 1.0560604724110431e-5, 1.1211133966439615e-6, 7.81262205501613e-8, 5.010904657519494e-9], [0.004607127392495961, 0.0001938291088798128, 2.3619023452776415e-5, 1.674074568243166e-6, 1.0779133252210256e-7], [0.015625, 0.00390625, 0.0009765625, 0.000244140625, 6.103515625e-5], [0.000244140625, 1.52587890625e-5, 9.5367431640625e-7, 5.960464477539063e-8, 3.725290298461914e-9]]
fkpl_coeffs_L2_test_ngrid_3_GLL.pdf
[[0.000162645849054975, 1.0465599140049831e-5, 6.729063532024829e-7, 4.2404287911779546e-8, 2.6540328192269194e-9], [0.0002401170164049581, 1.655450651195722e-5, 1.4475601598871342e-6, 1.1724960326123184e-7, 8.70653816670338e-9], [9.845888859211535e-5, 7.774320987060786e-6, 6.553611260568663e-7, 4.426571003517553e-8, 2.818766792141885e-9], [0.00013357704009381364, 7.463732920291641e-6, 7.771311681049361e-7, 5.4139239706282395e-8, 3.4697846705639162e-9], [9.803183492853563e-5, 6.277136810119457e-6, 5.515581454650211e-7, 4.0530125435562106e-8, 2.777002863641979e-9], [0.0003250746046973753, 1.912016983346137e-5, 3.308912953962959e-6, 2.5415135641422175e-7, 1.6755593824504363e-8], [0.015625, 0.00390625, 0.0009765625, 0.000244140625, 6.103515625e-5], [0.000244140625, 1.52587890625e-5, 9.5367431640625e-7, 5.960464477539063e-8, 3.725290298461914e-9]]
fkpl_conservation_test_ngrid_3_GLL.pdf
[[0.14664181770220974, 0.06546584218520007, 0.05541458710231074, 0.01974555428819708, 0.005687615069521508], [0.003973958788863551, 0.0013151792082636148, 0.0007241001038272485, 0.0002301430438426956, 6.145613936354202e-5], [4.166346993219358e-15, 5.036600558624537e-15, 1.290253196769475e-14, 4.0735789245218825e-14, 1.5966001363131454e-13], [1.9728084934773465e-6, 3.1975568273648396e-8, 1.831809922562343e-10, 4.4272339207844243e-11, 1.447120980544581e-12], [0.0005673588325043538, 0.00024192437846768007, 2.4223831206815676e-5, 1.7384082273717221e-6, 1.1287451524574988e-7], [0.015625, 0.00390625, 0.0009765625, 0.000244140625, 6.103515625e-5], [0.000244140625, 1.52587890625e-5, 9.5367431640625e-7, 5.960464477539063e-8, 3.725290298461914e-9]]
fkpl_timing_test_ngrid_3_GLL.pdf
[[7.0, 18.0, 111.0, 495.0, 2223.0], [265.0, 1659.0, 11233.0, 81444.0, 543686.0], [64.0, 256.0, 1024.0, 4096.0, 16384.0], [512.0, 4096.0, 32768.0, 262144.0, 2.097152e6]]
made inputs
vpa: ngrid: 5 nelement: 8 Lvpa: 12.0
vperp: ngrid: 5 nelement: 4 Lvperp: 6.0
made inputs
vpa: ngrid: 5 nelement: 8 Lvpa: 12.0
vperp: ngrid: 5 nelement: 4 Lvperp: 6.0
setting up GL quadrature   15:14:31
beginning (boundary) weights calculation   15:14:31
finished (boundary) weights calculation   15:14:31
begin elliptic operator assignment   15:14:31
finished elliptic operator constructor assignment   15:14:31
finished LU decomposition initialisation   15:14:31
finished YY array calculation   15:14:31
maximum(d2fvpavperp_dvpa2_err): 0.11219307366816
maximum(d2fvpavperp_dvperp2_err): 0.8427071654129412
begin C calculation   15:14:31
made inputs
vpa: ngrid: 5 nelement: finished C calculation   15:14:31
H boundary data:
max(lower_vpa_err) = 1.250711135686311e-6
max(upper_vpa_err) = 2.3182744028904523e-6
max(upper_vperp_err) = 1.3097068779499565e-6
dHdvpa boundary data:
max(lower_vpa_err) = 2.0124693084905276e-7
max(upper_vpa_err) = 1.1078000626066342e-6
max(upper_vperp_err) = 3.5506061818240883e-7
16 Lvpa: 12.0
dHdvperp boundary data:
max(lower_vpa_err) = 5.153964123972427e-7
max(upper_vpa_err) = vperp: ngrid: 5 nelement: 8 Lvperp: 6.0
1.671871873946268e-6
max(upper_vperp_err) = 2.054116713466614e-6
G boundary data:
max(lower_vpa_err) = 2.4486305184368007e-5
max(upper_vpa_err) = 8.920712110249696e-5
max(upper_vperp_err) = 7.004122472586971e-5
dGdvperp boundary data:
max(lower_vpa_err) = 4.6214119515997076e-5
max(upper_vpa_err) = 6.186503282601397e-5
max(upper_vperp_err) = 7.467293180396517e-5
d2Gdvperp2 boundary data:
max(lower_vpa_err) = 1.0074276587945796e-5
max(upper_vpa_err) = 1.8086797354710527e-5
max(upper_vperp_err) = 4.6543552250644105e-6
d2Gdvperpdvpa boundary data:
max(lower_vpa_err) = 3.7537915796859167e-6
max(upper_vpa_err) = 7.1056518480688835e-6
max(upper_vperp_err) = 4.8337176790022585e-6
d2Gdvpa2 boundary data:
max(lower_vpa_err) = 6.664542475456892e-7
max(upper_vpa_err) = 1.3785735933913301e-6
max(upper_vperp_err) = 2.2427413884640313e-6
maximum(H_M_err): 0.00030176691243466447 L2(H_M_err): 8.267423794935308e-6
maximum(dHdvpa_M_err): 0.0013244180287553253 L2(dHdvpa_M_err): 5.397546792484184e-5
maximum(dHdvperp_M_err): 0.004478732990902151 L2(dHdvperp_M_err): 9.165513989878578e-5
maximum(G_M_err): 0.00011702490491183326 L2(G_M_err): 3.827141460740127e-5
maximum(d2Gdvpa2_M_err): 0.00023494992626238442 L2(d2Gdvpa2_M_err): 1.1102485835043543e-5
maximum(dGdvperp_M_err): 0.0005340314482790987 L2(dGdvperp_M_err): 5.01855945677745e-5
maximum(d2Gdvperpdvpa_M_err): 0.0004044588054426361 L2(d2Gdvperpdvpa_M_err): 1.864496795395299e-5
maximum(d2Gdvperp2_M_err): 0.0007671465135949962 L2(d2Gdvperp2_M_err): 2.769313380890259e-5
maximum(C_M_err): 0.13852545236398398 L2(C_M_err): 0.0024603098726774667
dSdt: 0.0003395250637094192 should be >0.0
delta_n: -7.135375943112461e-15
delta_upar: 5.6844475431595144e-6
delta_pressure: 8.665372141280779e-5
made inputs
vpa: ngrid: 5 nelement: 16 Lvpa: 12.0
vperp: ngrid: 5 nelement: 8 Lvperp: 6.0
setting up GL quadrature   15:14:31
beginning (boundary) weights calculation   15:14:31
finished (boundary) weights calculation   15:14:33
begin elliptic operator assignment   15:14:33
finished elliptic operator constructor assignment   15:14:33
finished LU decomposition initialisation   15:14:33
finished YY array calculation   15:14:33
maximum(d2fvpavperp_dvpa2_err): 0.001530306391981484
maximum(d2fvpavperp_dvperp2_err): 0.19354033782475888
begin C calculation   15:14:33
finished C calculation   15:14:33
H boundary data:
max(lower_vpa_err) = 2.0248975496350852e-8
max(upper_vpa_err) = 2.927859837908997e-8
max(upper_vperp_err) = 2.2062522864185752e-8
dHdvpa boundary data:
max(lower_vpa_err) = 3.102063106386055e-9
max(upper_vpa_err) = 6.563723030372781e-9
max(upper_vperp_err) = 1.364009852983239e-9
dHdvperp boundary data:
max(lower_vpa_err) = 7.291533935319117e-8
max(upper_vpa_err) = 1.4561553543686323e-7
max(upper_vperp_err) = made inputs
vpa: ngrid: 5 nelement: 32 Lvpa: 12.0
vperp: ngrid: 5 nelement: 16 Lvperp: 6.0
2.4891693521308933e-7
G boundary data:
max(lower_vpa_err) = 1.2397879185499505e-6
max(upper_vpa_err) = 1.0480128835865798e-6
max(upper_vperp_err) = 1.2397879185499505e-6
dGdvperp boundary data:
max(lower_vpa_err) = 5.9302478292444505e-6
max(upper_vpa_err) = 6.9899921779637e-6
max(upper_vperp_err) = 9.03211198888254e-6
d2Gdvperp2 boundary data:
max(lower_vpa_err) = 1.308518969567185e-6
max(upper_vpa_err) = 1.840856089951881e-6
max(upper_vperp_err) = 5.662285593455163e-7
d2Gdvperpdvpa boundary data:
max(lower_vpa_err) = 4.998436504283221e-7
max(upper_vpa_err) = 6.978403086776996e-7
max(upper_vperp_err) = 5.726848005538354e-7
d2Gdvpa2 boundary data:
max(lower_vpa_err) = 6.482958898879065e-9
max(upper_vpa_err) = 1.071649759210036e-8
max(upper_vperp_err) = 2.183564176694297e-8
maximum(H_M_err): 3.2307878661264056e-6 L2(H_M_err): 1.4759478084158561e-7
maximum(dHdvpa_M_err): 2.6877539530012484e-5 L2(dHdvpa_M_err): 8.963500670324541e-7
maximum(dHdvperp_M_err): 7.788624189868087e-5 L2(dHdvperp_M_err): 1.4331325405348176e-6
maximum(G_M_err): 1.2397879185499505e-6 L2(G_M_err): 7.386176435456055e-7
maximum(d2Gdvpa2_M_err): 4.178327956849692e-6 L2(d2Gdvpa2_M_err): 1.821946351843197e-7
maximum(dGdvperp_M_err): 2.682324361623356e-5 L2(dGdvperp_M_err): 5.722789550834238e-6
maximum(d2Gdvperpdvpa_M_err): 1.4771340498211452e-5 L2(d2Gdvperpdvpa_M_err): 5.171597902142012e-7
maximum(d2Gdvperp2_M_err): 2.7551494087285633e-5 L2(d2Gdvperp2_M_err): 6.002712904357497e-7
maximum(C_M_err): 0.010526301815059582 L2(C_M_err): 0.0001763571352607852
dSdt: 9.701390531776414e-7 should be >0.0
delta_n: -5.681038351680817e-15
delta_upar: 1.4330830109698342e-8
delta_pressure: 5.016552856231658e-7
made inputs
vpa: ngrid: 5 nelement: 32 Lvpa: 12.0
vperp: ngrid: 5 nelement: 16 Lvperp: 6.0
setting up GL quadrature   15:14:33
beginning (boundary) weights calculation   15:14:33
finished (boundary) weights calculation   15:14:48
begin elliptic operator assignment   15:14:48
finished elliptic operator constructor assignment   15:14:48
finished LU decomposition initialisation   15:14:49
finished YY array calculation   15:14:49
maximum(d2fvpavperp_dvpa2_err): 0.00038407194077727524
maximum(d2fvpavperp_dvperp2_err): 0.017403574160740565
begin C calculation   15:14:49
finished C calculation   15:14:49
H boundary data:
max(lower_vpa_err) = 2.8293201115303646e-12
max(upper_vpa_err) = 3.208044940805621e-12
max(upper_vperp_err) = 3.9372671789550395e-12
dHdvpa boundary data:
max(lower_vpa_err) = 1.0137307659974226e-12
max(upper_vpa_err) = 1.1857834159023639e-11
max(upper_vperp_err) = 1.1449782094663519e-12
dHdvperp boundary data:
max(lower_vpa_err) = 4.226264060935847e-10
max(upper_vpa_err) = 8.472073580872985e-10
made inputsmax(upper_vperp_err) = 1.4340353522712679e-9
G boundary data:
max(lower_vpa_err) = 2.0792434440863872e-10
max(upper_vpa_err) = 1.8339996188387886e-10

max(upper_vperp_err) = vpa: ngrid: 2.0792434440863872e-10
5dGdvperp boundary data:
max(lower_vpa_err) =  nelement: 643.428557415752209e-8
max(upper_vpa_err) =  Lvpa: 12.0
4.041790568898307e-8
vperp: ngrid: 5 nelement: max(upper_vperp_err) = 5.221828447421473e-832 Lvperp: 
6.0
d2Gdvperp2 boundary data:
max(lower_vpa_err) = 7.59990553822476e-9
max(upper_vpa_err) = 1.0718390092145924e-8
max(upper_vperp_err) = 3.2637960806214394e-9
d2Gdvperpdvpa boundary data:
max(lower_vpa_err) = 2.8997639844874357e-9
max(upper_vpa_err) = 4.052691501832406e-9
max(upper_vperp_err) = 3.3154010647962195e-9
d2Gdvpa2 boundary data:
max(lower_vpa_err) = 7.515172512073676e-12
max(upper_vpa_err) = 2.1311633013887388e-12
max(upper_vperp_err) = 6.022203569155948e-12
maximum(H_M_err): 2.5505506706657854e-7 L2(H_M_err): 7.13744943848326e-9
maximum(dHdvpa_M_err): 5.896174274933053e-7 L2(dHdvpa_M_err): 1.7318373442448854e-8
maximum(dHdvperp_M_err): 7.5914310287139575e-6 L2(dHdvperp_M_err): 4.3767999205317684e-8
maximum(G_M_err): 6.676279684647568e-8 L2(G_M_err): 2.1944678190264928e-9
maximum(d2Gdvpa2_M_err): 1.2128169113445608e-7 L2(d2Gdvpa2_M_err): 3.253639975356406e-9
maximum(dGdvperp_M_err): 1.6397256108295621e-6 L2(dGdvperp_M_err): 3.411677146428303e-8
maximum(d2Gdvperpdvpa_M_err): 9.762892772442744e-7 L2(d2Gdvperpdvpa_M_err): 7.958229929508821e-9
maximum(d2Gdvperp2_M_err): 5.226947566994156e-7 L2(d2Gdvperp2_M_err): 7.314099332984964e-9
maximum(C_M_err): 0.0007703876600587945 L2(C_M_err): 1.4572508516893565e-5
dSdt: 2.7114871995128076e-9 should be >0.0
delta_n: 1.1935587021575068e-15
delta_upar: -9.980163825248475e-11
delta_pressure: 1.8374257563855105e-9
made inputs
vpa: ngrid: 5 nelement: 64 Lvpa: 12.0
vperp: ngrid: 5 nelement: 32 Lvperp: 6.0
setting up GL quadrature   15:14:49
beginning (boundary) weights calculation   15:14:49
finished (boundary) weights calculation   15:16:27
begin elliptic operator assignment   15:16:27
finished elliptic operator constructor assignment   15:16:27
finished LU decomposition initialisation   15:16:30
finished YY array calculation   15:16:30
maximum(d2fvpavperp_dvpa2_err): 3.774714809123836e-5
maximum(d2fvpavperp_dvperp2_err): 0.0012291091339220461
begin C calculation   15:16:30
finished C calculation   15:16:30
H boundary data:
max(lower_vpa_err) = 1.2886913758336505e-13
max(upper_vpa_err) = 9.50794998288984e-13
max(upper_vperp_err) = 1.2090328738167955e-13
dHdvpa boundary data:
max(lower_vpa_err) = 6.760217385881617e-13
max(upper_vpa_err) = 1.2497343437889441e-11
max(upper_vperp_err) = 1.3015578198549704e-12
dHdvperp boundary data:
max(lower_vpa_err) = 3.75983619238518e-12
max(upper_vpa_err) = 7.480639371837405e-12
max(upper_vperp_err) = 1.2720401121324443e-11
G boundary data:
max(lower_vpa_err) = 1.326938559031987e-12
max(upper_vpa_err) = 5.416112003331364e-12
max(upper_vperp_err) = 5.396572078097961e-12
dGdvperp boundary data:
max(lower_vpa_err) = 3.045073082574845e-10
max(upper_vpa_err) = 3.5864200498281207e-10
max(upper_vperp_err) = 4.6372272688444127e-10
d2Gdvperp2 boundary data:
max(lower_vpa_err) = 1.3707651680405775e-10
max(upper_vpa_err) = 1.211507560938685e-10
max(upper_vperp_err) = 2.8937130469586236e-11
d2Gdvperpdvpa boundary data:
max(lower_vpa_err) = 2.8785550332255383e-11
max(upper_vpa_err) = 2.854698422094515e-11
max(upper_vperp_err) = 3.363513634280224e-11
d2Gdvpa2 boundary data:
max(lower_vpa_err) = 7.822186041256396e-12
max(upper_vpa_err) = 1.170173333231439e-12
max(upper_vperp_err) = 6.942751928917801e-12
made inputs
vpa: ngrid: 5maximum(H_M_err):  nelement: 128 Lvpa: 7.090690035127523e-9 L2(H_M_err): 1.4775174414474762e-10
12.0
vperp: ngrid: 5 nelement: 64 Lvperp: 6.0
maximum(dHdvpa_M_err): 1.2045706709873372e-8 L2(dHdvpa_M_err): 2.858868434405569e-10
maximum(dHdvperp_M_err): 2.9015964840561903e-7 L2(dHdvperp_M_err): 8.623152613239594e-10
maximum(G_M_err): 1.5025343191865659e-9 L2(G_M_err): 4.0805552171807536e-11
maximum(d2Gdvpa2_M_err): 2.7111488609676826e-9 L2(d2Gdvpa2_M_err): 5.497953003465016e-11
maximum(dGdvperp_M_err): 5.9060396401591775e-8 L2(dGdvperp_M_err): 3.4952890682121313e-10
maximum(d2Gdvperpdvpa_M_err): 3.5571510969851367e-8 L2(d2Gdvperpdvpa_M_err): 1.5029357751340919e-10
maximum(d2Gdvperp2_M_err): 8.524436934109758e-9 L2(d2Gdvperp2_M_err): 1.2691064547512596e-10
maximum(C_M_err): 0.0001980801741361449 L2(C_M_err): 2.0449269785605596e-6
dSdt: 1.0367701087762325e-10 should be >0.0
delta_n: 4.1933637946769476e-14
delta_upar: 4.873813373146715e-12
delta_pressure: 4.536217490105059e-12
made inputs
vpa: ngrid: 5 nelement: 128 Lvpa: 12.0
vperp: ngrid: 5 nelement: 64 Lvperp: 6.0
setting up GL quadrature   15:16:30
beginning (boundary) weights calculation   15:16:30
finished (boundary) weights calculation   15:29:41
begin elliptic operator assignment   15:29:42
finished elliptic operator constructor assignment   15:29:44
finished LU decomposition initialisation   15:29:55
finished YY array calculation   15:29:55
maximum(d2fvpavperp_dvpa2_err): 2.651029539224581e-6
maximum(d2fvpavperp_dvperp2_err): 7.96254926118678e-5
begin C calculation   15:29:55
finished C calculation   15:29:57
H boundary data:
max(lower_vpa_err) = 8.762435221854048e-14
max(upper_vpa_err) = 1.256855730602524e-12
max(upper_vperp_err) = 1.5246137685664962e-13
dHdvpa boundary data:
max(lower_vpa_err) = 6.455287693274414e-13
max(upper_vpa_err) = 1.2566059304219834e-11
max(upper_vperp_err) = 1.3147382488254422e-12
dHdvperp boundary data:
max(lower_vpa_err) = 8.628061576288447e-14
max(upper_vpa_err) = 5.684159740115824e-13
max(upper_vperp_err) = 4.745162596186958e-14
G boundary data:
max(lower_vpa_err) = 1.2079226507921703e-11
max(upper_vpa_err) = 5.384137580222159e-12
max(upper_vperp_err) = 1.1992185022791091e-11
dGdvperp boundary data:
max(lower_vpa_err) = 1.4957789781083752e-12
max(upper_vpa_err) = 8.217870828275409e-13
max(upper_vperp_err) = 1.5173418077552014e-12
d2Gdvperp2 boundary data:
max(lower_vpa_err) = 2.774302731989309e-10
max(upper_vpa_err) = 1.0642658976323105e-10
max(upper_vperp_err) = 1.0418055307326313e-13
d2Gdvperpdvpa boundary data:
max(lower_vpa_err) = 3.578602691955979e-12
max(upper_vpa_err) = 7.64345531312216e-12
max(upper_vperp_err) = 7.643975730164954e-12
d2Gdvpa2 boundary data:
max(lower_vpa_err) = 7.809290106935984e-12
max(upper_vpa_err) = 1.1247443948425584e-12
max(upper_vperp_err) = 7.005396263082275e-12
maximum(H_M_err): 1.3341749927064939e-10 L2(H_M_err): 3.2897470135307176e-12
maximum(dHdvpa_M_err): 2.0928531130337547e-10 L2(dHdvpa_M_err): 4.759751019489176e-12
maximum(dHdvperp_M_err): 9.511461172492797e-9 L2(dHdvperp_M_err): 1.7582568607163254e-11
maximum(G_M_err): 3.625277855690001e-11 L2(G_M_err): 2.1368460638353405e-11
maximum(d2Gdvpa2_M_err): 5.179390250020788e-11 L2(d2Gdvpa2_M_err): 5.406326557061539e-12
maximum(dGdvperp_M_err): 1.9104650587215e-9 L2(dGdvperp_M_err): 6.637818392290019e-12
maximum(d2Gdvperpdvpa_M_err): 1.1537180319171753e-9 L2(d2Gdvperpdvpa_M_err): 5.364548141801808e-12
maximum(d2Gdvperp2_M_err): 2.774302731989309e-10 L2(d2Gdvperp2_M_err): 2.1075667742184062e-12
maximum(C_M_err): 1.949121667842828e-5 L2(C_M_err): 1.5916566370757255e-7
dSdt: -2.354059704099387e-12 should be >0.0
delta_n: 1.8037024486332233e-13
delta_upar: 5.850130244048564e-12
delta_pressure: -4.5028087363727996e-12
fkpl_C_G_H_max_test_ngrid_5_GLL.pdf
[[0.13852545236398398, 0.010526301815059582, 0.0007703876600587945, 0.0001980801741361449, 1.949121667842828e-5], [0.00030176691243466447, 3.2307878661264056e-6, 2.5505506706657854e-7, 7.090690035127523e-9, 1.3341749927064939e-10], [0.00011702490491183326, 1.2397879185499505e-6, 6.676279684647568e-8, 1.5025343191865659e-9, 3.625277855690001e-11], [0.00390625, 0.000244140625, 1.52587890625e-5, 9.5367431640625e-7, 5.960464477539063e-8], [0.000244140625, 3.814697265625e-6, 5.960464477539063e-8, 9.313225746154785e-10, 1.4551915228366852e-11]]
fkpl_coeffs_max_test_ngrid_5_GLL.pdf
[[0.0013244180287553253, 2.6877539530012484e-5, 5.896174274933053e-7, 1.2045706709873372e-8, 2.0928531130337547e-10], [0.004478732990902151, 7.788624189868087e-5, 7.5914310287139575e-6, 2.9015964840561903e-7, 9.511461172492797e-9], [0.0007671465135949962, 2.7551494087285633e-5, 5.226947566994156e-7, 8.524436934109758e-9, 2.774302731989309e-10], [0.00023494992626238442, 4.178327956849692e-6, 1.2128169113445608e-7, 2.7111488609676826e-9, 5.179390250020788e-11], [0.0004044588054426361, 1.4771340498211452e-5, 9.762892772442744e-7, 3.5571510969851367e-8, 1.1537180319171753e-9], [0.0005340314482790987, 2.682324361623356e-5, 1.6397256108295621e-6, 5.9060396401591775e-8, 1.9104650587215e-9], [0.00390625, 0.000244140625, 1.52587890625e-5, 9.5367431640625e-7, 5.960464477539063e-8], [0.000244140625, 3.814697265625e-6, 5.960464477539063e-8, 9.313225746154785e-10, 1.4551915228366852e-11]]
fkpl_C_G_H_L2_test_ngrid_5_GLL.pdf
[[0.0024603098726774667, 0.0001763571352607852, 1.4572508516893565e-5, 2.0449269785605596e-6, 1.5916566370757255e-7], [8.267423794935308e-6, 1.4759478084158561e-7, 7.13744943848326e-9, 1.4775174414474762e-10, 3.2897470135307176e-12], [3.827141460740127e-5, 7.386176435456055e-7, 2.1944678190264928e-9, 4.0805552171807536e-11, 2.1368460638353405e-11], [0.00390625, 0.000244140625, 1.52587890625e-5, 9.5367431640625e-7, 5.960464477539063e-8], [0.000244140625, 3.814697265625e-6, 5.960464477539063e-8, 9.313225746154785e-10, 1.4551915228366852e-11]]
fkpl_coeffs_L2_test_ngrid_5_GLL.pdf
[[5.397546792484184e-5, 8.963500670324541e-7, 1.7318373442448854e-8, 2.858868434405569e-10, 4.759751019489176e-12], [9.165513989878578e-5, 1.4331325405348176e-6, 4.3767999205317684e-8, 8.623152613239594e-10, 1.7582568607163254e-11], [2.769313380890259e-5, 6.002712904357497e-7, 7.314099332984964e-9, 1.2691064547512596e-10, 2.1075667742184062e-12], [1.1102485835043543e-5, 1.821946351843197e-7, 3.253639975356406e-9, 5.497953003465016e-11, 5.406326557061539e-12], [1.864496795395299e-5, 5.171597902142012e-7, 7.958229929508821e-9, 1.5029357751340919e-10, 5.364548141801808e-12], [5.01855945677745e-5, 5.722789550834238e-6, 3.411677146428303e-8, 3.4952890682121313e-10, 6.637818392290019e-12], [0.00390625, 0.000244140625, 1.52587890625e-5, 9.5367431640625e-7, 5.960464477539063e-8], [0.000244140625, 3.814697265625e-6, 5.960464477539063e-8, 9.313225746154785e-10, 1.4551915228366852e-11]]
fkpl_conservation_test_ngrid_5_GLL.pdf
[[0.13852545236398398, 0.010526301815059582, 0.0007703876600587945, 0.0001980801741361449, 1.949121667842828e-5], [0.0024603098726774667, 0.0001763571352607852, 1.4572508516893565e-5, 2.0449269785605596e-6, 1.5916566370757255e-7], [7.135375943112461e-15, 5.681038351680817e-15, 1.1935587021575068e-15, 4.1933637946769476e-14, 1.8037024486332233e-13], [5.6844475431595144e-6, 1.4330830109698342e-8, 9.980163825248475e-11, 4.873813373146715e-12, 5.850130244048564e-12], [8.665372141280779e-5, 5.016552856231658e-7, 1.8374257563855105e-9, 4.536217490105059e-12, 4.5028087363727996e-12], [0.00390625, 0.000244140625, 1.52587890625e-5, 9.5367431640625e-7, 5.960464477539063e-8], [0.000244140625, 3.814697265625e-6, 5.960464477539063e-8, 9.313225746154785e-10, 1.4551915228366852e-11]]
fkpl_timing_test_ngrid_5_GLL.pdf
[[6.0, 83.0, 146.0, 496.0, 2427.0], [297.0, 1895.0, 15471.0, 100726.0, 804572.0], [16.0, 64.0, 256.0, 1024.0, 4096.0], [64.0, 512.0, 4096.0, 32768.0, 262144.0]]
made inputs
vpa: ngrid: 7 nelement: 4 Lvpa: 12.0
vperp: ngrid: 7 nelement: 2 Lvperp: 6.0
made inputs
vpa: ngrid: 7 nelement: 4 Lvpa: 12.0
vperp: ngrid: 7 nelement: 2 Lvperp: 6.0
setting up GL quadrature   15:29:58
beginning (boundary) weights calculation   15:29:58
finished (boundary) weights calculation   15:29:59
begin elliptic operator assignment   15:29:59
finished elliptic operator constructor assignment   15:29:59
finished LU decomposition initialisation   15:29:59
finished YY array calculation   15:29:59
maximum(d2fvpavperp_dvpa2_err): 0.16532345677550775
maximum(d2fvpavperp_dvperp2_err): 1.6071952912520038
begin C calculation   15:29:59
made inputs
vpa: ngrid: 7 nelement: 8 Lvpa: 12.0
vperp: ngrid: 7 nelement: 4 Lvperp: 6.0
finished C calculation   15:29:59
H boundary data:
max(lower_vpa_err) = 2.7275610522126126e-6
max(upper_vpa_err) = 8.184971211078196e-7
max(upper_vperp_err) = 2.773440952258399e-6
dHdvpa boundary data:
max(lower_vpa_err) = 7.132225954727123e-7
max(upper_vpa_err) = 2.4970818892772195e-6
max(upper_vperp_err) = 5.343585941699341e-7
dHdvperp boundary data:
max(lower_vpa_err) = 2.96691836516004e-6
max(upper_vpa_err) = 4.784630201240883e-6
max(upper_vperp_err) = 1.1959097412011371e-5
G boundary data:
max(lower_vpa_err) = 4.877436281525149e-5
max(upper_vpa_err) = 0.00013391310967669767
max(upper_vperp_err) = 0.00013391310967669767
dGdvperp boundary data:
max(lower_vpa_err) = 0.00025212397179064094
max(upper_vpa_err) = 0.000286690802464018
max(upper_vperp_err) = 0.00038250575457665636
d2Gdvperp2 boundary data:
max(lower_vpa_err) = 5.3393480420688144e-5
max(upper_vpa_err) = 6.582456857237573e-5
max(upper_vperp_err) = 2.55928574467712e-5
d2Gdvperpdvpa boundary data:
max(lower_vpa_err) = 2.0214125934329974e-5
max(upper_vpa_err) = 2.4676017593178168e-5
max(upper_vperp_err) = 2.417966173590136e-5
d2Gdvpa2 boundary data:
max(lower_vpa_err) = 1.1180227250592867e-6
max(upper_vpa_err) = 2.746657483376422e-6
max(upper_vperp_err) = 4.199464472193215e-6
maximum(H_M_err): 0.0006309996680704533 L2(H_M_err): 2.557851849495637e-5
maximum(dHdvpa_M_err): 0.0052764667448848 L2(dHdvpa_M_err): 0.00026289565486593743
maximum(dHdvperp_M_err): 0.004425784011258066 L2(dHdvperp_M_err): 0.000224275524947831
maximum(G_M_err): 0.00028380475494560464 L2(G_M_err): 6.22412415898005e-5
maximum(d2Gdvpa2_M_err): 0.0005394803881073607 L2(d2Gdvpa2_M_err): 3.499242281710138e-5
maximum(dGdvperp_M_err): 0.0006371868410837561 L2(dGdvperp_M_err): 0.00024281123627496623
maximum(d2Gdvperpdvpa_M_err): 0.0009185912655483158 L2(d2Gdvperpdvpa_M_err): 6.975665085220292e-5
maximum(d2Gdvperp2_M_err): 0.0012111377025836312 L2(d2Gdvperp2_M_err): 6.099811764017731e-5
maximum(C_M_err): 0.2350036566973432 L2(C_M_err): 0.0063598766143691835
dSdt: 0.000629389462850099 should be >0.0
delta_n: -2.2809623623085514e-14
delta_upar: -9.050656489532895e-7
delta_pressure: 0.00019716262811926967
made inputs
vpa: ngrid: 7 nelement: 8 Lvpa: 12.0
vperp: ngrid: 7 nelement: 4 Lvperp: 6.0
setting up GL quadrature   15:29:59
beginning (boundary) weights calculation   15:29:59
finished (boundary) weights calculation   15:30:01
begin elliptic operator assignment   15:30:01
finished elliptic operator constructor assignment   15:30:01
finished LU decomposition initialisation   15:30:01
finished YY array calculation   15:30:01
maximum(d2fvpavperp_dvpa2_err): 0.010134848860970935
maximum(d2fvpavperp_dvperp2_err): 0.08038064921850196
begin C calculation   15:30:01
finished C calculation   15:30:01
H boundary data:
max(lower_vpa_err) = 8.973855269989173e-10
max(upper_vpa_err) = 3.404167364662314e-9
max(upper_vperp_err) = 1.6549541737020945e-9
dHdvpa boundary data:
max(lower_vpa_err) = 1.0824724450131384e-10
max(upper_vpa_err) = 9.308498832849565e-10
max(upper_vperp_err) = 1.7733194741190866e-10
dHdvperp boundary data:
max(lower_vpa_err) = 4.9773271143527253e-8
max(upper_vpa_err) = 9.74547950689103e-8
max(upper_vperp_err) = 1.7694867921261292e-7
G boundary data:
max(lower_vpa_err) = 1.0621286428147414e-7
max(upper_vpa_err) = 4.928780406032729e-8
max(upper_vperp_err) = 1.0621286428147414e-7
made inputs
vpa: ngrid: 7 nelement: 16 Lvpa: 12.0
vperp: ngrid: 7 nelement: 8 Lvperp: 6.0
dGdvperp boundary data:
max(lower_vpa_err) = 4.110376379484748e-6
max(upper_vpa_err) = 4.844129217529414e-6
max(upper_vperp_err) = 6.267019432515397e-6
d2Gdvperp2 boundary data:
max(lower_vpa_err) = 8.9659536842146e-7
max(upper_vpa_err) = 1.2405865639775193e-6
max(upper_vperp_err) = 3.994600522722047e-7
d2Gdvperpdvpa boundary data:
max(lower_vpa_err) = 3.4137225671249105e-7
max(upper_vpa_err) = 4.679343926228263e-7
max(upper_vperp_err) = 3.964068681083788e-7
d2Gdvpa2 boundary data:
max(lower_vpa_err) = 1.6146144837803078e-10
max(upper_vpa_err) = 1.4587075575223096e-9
max(upper_vperp_err) = 2.0187873039922977e-9
maximum(H_M_err): 5.516897222723749e-6 L2(H_M_err): 1.9523892441473391e-7
maximum(dHdvpa_M_err): 4.146873392771311e-5 L2(dHdvpa_M_err): 1.6010027189120426e-6
maximum(dHdvperp_M_err): 0.00034507692581000476 L2(dHdvperp_M_err): 3.3425208838805816e-6
maximum(G_M_err): 1.056496379714389e-6 L2(G_M_err): 6.819229121471981e-8
maximum(d2Gdvpa2_M_err): 5.2862297899025634e-6 L2(d2Gdvpa2_M_err): 2.1085474449983808e-7
maximum(dGdvperp_M_err): 4.2272953107921896e-5 L2(dGdvperp_M_err): 3.966908322927979e-6
maximum(d2Gdvperpdvpa_M_err): 2.945484815180914e-5 L2(d2Gdvperpdvpa_M_err): 5.254359042538814e-7
maximum(d2Gdvperp2_M_err): 1.825897956031497e-5 L2(d2Gdvperp2_M_err): 6.014942618171157e-7
maximum(C_M_err): 0.011447944515768412 L2(C_M_err): 0.00016785759287701746
dSdt: 6.156696898503948e-7 should be >0.0
delta_n: -4.8233979342169083e-14
delta_upar: 1.6868346947080208e-8
delta_pressure: 1.9194524261752296e-7
made inputs
vpa: ngrid: 7 nelement: 16 Lvpa: 12.0
vperp: ngrid: 7 nelement: 8 Lvperp: 6.0
setting up GL quadrature   15:30:01
beginning (boundary) weights calculation   15:30:01
finished (boundary) weights calculation   15:30:11
begin elliptic operator assignment   15:30:11
finished elliptic operator constructor assignment   15:30:11
finished LU decomposition initialisation   15:30:12
finished YY array calculation   15:30:12
maximum(d2fvpavperp_dvpa2_err): 2.679052933118875e-5
maximum(d2fvpavperp_dvperp2_err): 0.008731342864201075
begin C calculation   15:30:12
made inputsfinished C calculation   15:30:12
H boundary data:
max(lower_vpa_err) = 7.609746166536979e-13
max(upper_vpa_err) = 2.2285784329056924e-12
max(upper_vperp_err) = 8.58424442640171e-13
dHdvpa boundary data:
max(lower_vpa_err) = 5.484883380812988e-13
max(upper_vpa_err) = 1.277223465878663e-11
max(upper_vperp_err) = 1.3614057797761703e-12
dHdvperp boundary data:
max(lower_vpa_err) = 5.906633949309681e-10
max(upper_vpa_err) = 1.1777766123488576e-9
max(upper_vperp_err) = 
2.0228997747728794e-9
vpa: ngrid: 7 nelement: 32G boundary data:
max(lower_vpa_err) =  Lvpa: 5.296740823723667e-11
max(upper_vpa_err) = 12.0
4.077271853475395e-11
max(upper_vperp_err) = 5.296740823723667e-11vperp: ngrid: 7 nelement: 16
dGdvperp boundary data:
 Lvperp: 6.0
max(lower_vpa_err) = 4.808641207443287e-8
max(upper_vpa_err) = 5.6675263371275264e-8
max(upper_vperp_err) = 7.325257800339102e-8
d2Gdvperp2 boundary data:
max(lower_vpa_err) = 1.0619563201030857e-8
max(upper_vpa_err) = 1.4914725926518102e-8
max(upper_vperp_err) = 4.596540592594067e-9
d2Gdvperpdvpa boundary data:
max(lower_vpa_err) = 4.0520534913546236e-9
max(upper_vpa_err) = 5.637329061891627e-9
max(upper_vperp_err) = 4.646963307208818e-9
d2Gdvpa2 boundary data:
max(lower_vpa_err) = 7.77897104370373e-12
max(upper_vpa_err) = 1.1202323790815427e-12
max(upper_vperp_err) = 7.229751519677308e-12
maximum(H_M_err): 3.05412770629232e-8 L2(H_M_err): 1.583769827085239e-9
maximum(dHdvpa_M_err): 1.3835969109310398e-7 L2(dHdvpa_M_err): 5.045486743191758e-9
maximum(dHdvperp_M_err): 4.2823803586208853e-7 L2(dHdvperp_M_err): 5.0296939534342656e-9
maximum(G_M_err): 3.3513956054775917e-9 L2(G_M_err): 1.81319724243487e-10
maximum(d2Gdvpa2_M_err): 2.981354296771599e-8 L2(d2Gdvpa2_M_err): 1.2867768193286518e-9
maximum(dGdvperp_M_err): 1.7159963709711956e-7 L2(dGdvperp_M_err): 4.611610305877534e-8
maximum(d2Gdvperpdvpa_M_err): 1.003364159431197e-7 L2(d2Gdvperpdvpa_M_err): 3.163796723915407e-9
maximum(d2Gdvperp2_M_err): 2.749673756463622e-7 L2(d2Gdvperp2_M_err): 4.479028536187569e-9
maximum(C_M_err): 0.00031521966596359443 L2(C_M_err): 3.540267859160749e-6
dSdt: -4.681692712081668e-10 should be >0.0
delta_n: -1.1172063209852497e-13
delta_upar: -1.4140652289936854e-10
delta_pressure: -4.678065890826753e-10
made inputs
vpa: ngrid: 7 nelement: 32 Lvpa: 12.0
vperp: ngrid: 7 nelement: 16 Lvperp: 6.0
setting up GL quadrature   15:30:12
beginning (boundary) weights calculation   15:30:12
finished (boundary) weights calculation   15:31:25
begin elliptic operator assignment   15:31:26
finished elliptic operator constructor assignment   15:31:26
finished LU decomposition initialisation   15:31:28
finished YY array calculation   15:31:28
maximum(d2fvpavperp_dvpa2_err): 2.4124181894347174e-6
maximum(d2fvpavperp_dvperp2_err): 0.00020527576803308634
begin C calculation   15:31:28
finished C calculation   15:31:29
H boundary data:
max(lower_vpa_err) = 6.747380432159389e-14
max(upper_vpa_err) = 1.2381207170619746e-12
max(upper_vperp_err) = 1.329492071988625e-13
dHdvpa boundary data:
max(lower_vpa_err) = 6.454281553658348e-13
max(upper_vpa_err) = 1.2566739315822417e-11
max(upper_vperp_err) = 1.3149429461956075e-12
dHdvperp boundary data:
max(lower_vpa_err) = 4.622222743444482e-13
max(upper_vpa_err) = 1.0878832557015272e-12
max(upper_vperp_err) = 1.571746205408786e-12
G boundary data:
max(lower_vpa_err) = 1.06332720406499e-11
max(upper_vpa_err) = 4.7055692675712635e-12
max(upper_vperp_err) = 1.0629719326971099e-11
dGdvperp boundary data:
max(lower_vpa_err) = 3.769529133279548e-11
max(upper_vpa_err) = 4.479716597671768e-11
max(upper_vperp_err) = 5.7470583847418766e-11
d2Gdvperp2 boundary data:
max(lower_vpa_err) = 6.000144825435427e-11made inputs
max(upper_vpa_err) = 1.2539552729506909e-11
max(upper_vperp_err) = 3.6013414472790828e-12
d2Gdvperpdvpa boundary data:
max(lower_vpa_err) = 9.085440733080929e-13
max(upper_vpa_err) = 1.2088885448235942e-11

max(upper_vperp_err) = 1.1355742735030816e-11
d2Gdvpa2 boundary data:
max(lower_vpa_err) = 7.808808287490532e-12
vpa: ngrid: 7 nelement: 64 Lvpa: max(upper_vpa_err) = 1.1248675602093527e-12
12.0
max(upper_vperp_err) = 7.005701574414047e-12
vperp: ngrid: 7 nelement: 32 Lvperp: 6.0
maximum(H_M_err): 3.8896441623137434e-10 L2(H_M_err): 1.0738513361859409e-11
maximum(dHdvpa_M_err): 9.426428526637665e-10 L2(dHdvpa_M_err): 2.7737578941627416e-11
maximum(dHdvperp_M_err): 2.6870052934405142e-8 L2(dHdvperp_M_err): 9.147114024284572e-11
maximum(G_M_err): 7.781730815281662e-11 L2(G_M_err): 8.970934062374587e-12
maximum(d2Gdvpa2_M_err): 1.9131984885234488e-10 L2(d2Gdvpa2_M_err): 7.169328475118158e-12
maximum(dGdvperp_M_err): 4.1565181331526535e-9 L2(dGdvperp_M_err): 3.941666656043486e-11
maximum(d2Gdvperpdvpa_M_err): 2.709243512692694e-9 L2(d2Gdvperpdvpa_M_err): 1.2732042959175515e-11
maximum(d2Gdvperp2_M_err): 1.4583071417106908e-9 L2(d2Gdvperp2_M_err): 1.3023476409455556e-11
maximum(C_M_err): 3.903808171056644e-6 L2(C_M_err): 6.781149979275991e-8
dSdt: -2.6068813526506185e-13 should be >0.0
delta_n: -3.5376822366963917e-13
delta_upar: 5.530435305548463e-12
delta_pressure: -4.75703416711213e-13
made inputs
vpa: ngrid: 7 nelement: 64 Lvpa: 12.0
vperp: ngrid: 7 nelement: 32 Lvperp: 6.0
setting up GL quadrature   15:31:29
beginning (boundary) weights calculation   15:31:29
finished (boundary) weights calculation   15:45:51
begin elliptic operator assignment   15:45:53
finished elliptic operator constructor assignment   15:45:56
finished LU decomposition initialisation   15:46:06
finished YY array calculation   15:46:06
maximum(d2fvpavperp_dvpa2_err): 5.8925084811534134e-8
maximum(d2fvpavperp_dvperp2_err): 3.623227686544439e-6
begin C calculation   15:46:06
finished C calculation   15:46:08
H boundary data:
max(lower_vpa_err) = 7.943645741192995e-14
max(upper_vpa_err) = 1.2466416787759727e-12
max(upper_vperp_err) = 1.4449552665496412e-13
dHdvpa boundary data:
max(lower_vpa_err) = 6.45636322182952e-13
max(upper_vpa_err) = 1.256697523821515e-11
max(upper_vperp_err) = 1.3147243710376344e-12
dHdvperp boundary data:
max(lower_vpa_err) = 8.683131253947596e-14
max(upper_vpa_err) = 5.857033608114293e-13
max(upper_vperp_err) = 2.5732888042639956e-14
G boundary data:
max(lower_vpa_err) = 1.1359801987964602e-11
max(upper_vpa_err) = 5.027089855502709e-12
max(upper_vperp_err) = 1.1359801987964602e-11
dGdvperp boundary data:
max(lower_vpa_err) = 1.4485383378892713e-12
max(upper_vpa_err) = 9.698908343125368e-13
max(upper_vperp_err) = 9.722223026642496e-13
d2Gdvperp2 boundary data:
max(lower_vpa_err) = 2.6773111505562497e-10
max(upper_vpa_err) = 1.032442464854455e-10
max(upper_vperp_err) = 6.340761249390425e-14
d2Gdvperpdvpa boundary data:
max(lower_vpa_err) = 3.4532585124757986e-12
max(upper_vpa_err) = 7.789678624359198e-12
max(upper_vperp_err) = 7.789678624359198e-12
d2Gdvpa2 boundary data:
max(lower_vpa_err) = 7.80875711314799e-12
max(upper_vpa_err) = 1.1249542963831516e-12
max(upper_vperp_err) = 7.004376245678401e-12
maximum(H_M_err): 7.69584396209666e-12 L2(H_M_err): 1.6169046430390734e-12
maximum(dHdvpa_M_err): 1.256697523821515e-11 L2(dHdvpa_M_err): 1.4332150589451027e-12
maximum(dHdvperp_M_err): 2.7200479195410576e-10 L2(dHdvperp_M_err): 4.715193418368361e-13
maximum(G_M_err): 2.9045210681033495e-11 L2(G_M_err): 1.7199602033693805e-11
maximum(d2Gdvpa2_M_err): 7.80875711314799e-12 L2(d2Gdvpa2_M_err): 5.285131556889585e-12
maximum(dGdvperp_M_err): 3.9563697804001485e-11 L2(dGdvperp_M_err): 4.693790632071181e-12
maximum(d2Gdvperpdvpa_M_err): 2.6068204452694976e-11 L2(d2Gdvperpdvpa_M_err): 4.470070881020146e-12
maximum(d2Gdvperp2_M_err): 2.6773111505562497e-10 L2(d2Gdvperp2_M_err): 5.226452422889844e-13
maximum(C_M_err): 3.0144649299779144e-7 L2(C_M_err): 2.6822453624849816e-9
dSdt: -1.517900842662775e-12 should be >0.0
delta_n: -1.2993172174775191e-12
delta_upar: 4.382716084666107e-12
delta_pressure: -3.833982116380302e-12
fkpl_C_G_H_max_test_ngrid_7_GLL.pdf
[[0.2350036566973432, 0.011447944515768412, 0.00031521966596359443, 3.903808171056644e-6, 3.0144649299779144e-7], [0.0006309996680704533, 5.516897222723749e-6, 3.05412770629232e-8, 3.8896441623137434e-10, 7.69584396209666e-12], [0.00028380475494560464, 1.056496379714389e-6, 3.3513956054775917e-9, 7.781730815281662e-11, 2.9045210681033495e-11], [0.015625, 0.000244140625, 3.814697265625e-6, 5.960464477539063e-8, 9.313225746154785e-10], [0.00390625, 1.52587890625e-5, 5.960464477539063e-8, 2.3283064365386963e-10, 9.094947017729282e-13]]
fkpl_coeffs_max_test_ngrid_7_GLL.pdf
[[0.0052764667448848, 4.146873392771311e-5, 1.3835969109310398e-7, 9.426428526637665e-10, 1.256697523821515e-11], [0.004425784011258066, 0.00034507692581000476, 4.2823803586208853e-7, 2.6870052934405142e-8, 2.7200479195410576e-10], [0.0012111377025836312, 1.825897956031497e-5, 2.749673756463622e-7, 1.4583071417106908e-9, 2.6773111505562497e-10], [0.0005394803881073607, 5.2862297899025634e-6, 2.981354296771599e-8, 1.9131984885234488e-10, 7.80875711314799e-12], [0.0009185912655483158, 2.945484815180914e-5, 1.003364159431197e-7, 2.709243512692694e-9, 2.6068204452694976e-11], [0.0006371868410837561, 4.2272953107921896e-5, 1.7159963709711956e-7, 4.1565181331526535e-9, 3.9563697804001485e-11], [0.015625, 0.000244140625, 3.814697265625e-6, 5.960464477539063e-8, 9.313225746154785e-10], [0.00390625, 1.52587890625e-5, 5.960464477539063e-8, 2.3283064365386963e-10, 9.094947017729282e-13]]
fkpl_C_G_H_L2_test_ngrid_7_GLL.pdf
[[0.0063598766143691835, 0.00016785759287701746, 3.540267859160749e-6, 6.781149979275991e-8, 2.6822453624849816e-9], [2.557851849495637e-5, 1.9523892441473391e-7, 1.583769827085239e-9, 1.0738513361859409e-11, 1.6169046430390734e-12], [6.22412415898005e-5, 6.819229121471981e-8, 1.81319724243487e-10, 8.970934062374587e-12, 1.7199602033693805e-11], [0.015625, 0.000244140625, 3.814697265625e-6, 5.960464477539063e-8, 9.313225746154785e-10], [0.00390625, 1.52587890625e-5, 5.960464477539063e-8, 2.3283064365386963e-10, 9.094947017729282e-13]]
fkpl_coeffs_L2_test_ngrid_7_GLL.pdf
[[0.00026289565486593743, 1.6010027189120426e-6, 5.045486743191758e-9, 2.7737578941627416e-11, 1.4332150589451027e-12], [0.000224275524947831, 3.3425208838805816e-6, 5.0296939534342656e-9, 9.147114024284572e-11, 4.715193418368361e-13], [6.099811764017731e-5, 6.014942618171157e-7, 4.479028536187569e-9, 1.3023476409455556e-11, 5.226452422889844e-13], [3.499242281710138e-5, 2.1085474449983808e-7, 1.2867768193286518e-9, 7.169328475118158e-12, 5.285131556889585e-12], [6.975665085220292e-5, 5.254359042538814e-7, 3.163796723915407e-9, 1.2732042959175515e-11, 4.470070881020146e-12], [0.00024281123627496623, 3.966908322927979e-6, 4.611610305877534e-8, 3.941666656043486e-11, 4.693790632071181e-12], [0.015625, 0.000244140625, 3.814697265625e-6, 5.960464477539063e-8, 9.313225746154785e-10], [0.00390625, 1.52587890625e-5, 5.960464477539063e-8, 2.3283064365386963e-10, 9.094947017729282e-13]]
fkpl_conservation_test_ngrid_7_GLL.pdf
[[0.2350036566973432, 0.011447944515768412, 0.00031521966596359443, 3.903808171056644e-6, 3.0144649299779144e-7], [0.0063598766143691835, 0.00016785759287701746, 3.540267859160749e-6, 6.781149979275991e-8, 2.6822453624849816e-9], [2.2809623623085514e-14, 4.8233979342169083e-14, 1.1172063209852497e-13, 3.5376822366963917e-13, 1.2993172174775191e-12], [9.050656489532895e-7, 1.6868346947080208e-8, 1.4140652289936854e-10, 5.530435305548463e-12, 4.382716084666107e-12], [0.00019716262811926967, 1.9194524261752296e-7, 4.678065890826753e-10, 4.75703416711213e-13, 3.833982116380302e-12], [0.015625, 0.000244140625, 3.814697265625e-6, 5.960464477539063e-8, 9.313225746154785e-10], [0.00390625, 1.52587890625e-5, 5.960464477539063e-8, 2.3283064365386963e-10, 9.094947017729282e-13]]
fkpl_timing_test_ngrid_7_GLL.pdf
[[7.0, 19.0, 66.0, 393.0, 2339.0], [376.0, 1639.0, 10976.0, 76100.0, 877246.0], [4.0, 16.0, 64.0, 256.0, 1024.0], [8.0, 64.0, 512.0, 4096.0, 32768.0]]
made inputs
vpa: ngrid: 9 nelement: 4 Lvpa: 12.0
vperp: ngrid: 9 nelement: 2 Lvperp: made inputs
vpa: ngrid: 9 nelement: 4 Lvpa: 12.0
vperp: ngrid: 9 nelement: 2 Lvperp: 6.0
6.0
setting up GL quadrature   15:46:11
beginning (boundary) weights calculation   15:46:11
finished (boundary) weights calculation   15:46:13
begin elliptic operator assignment   15:46:13
finished elliptic operator constructor assignment   15:46:13
finished LU decomposition initialisation   15:46:14
finished YY array calculation   15:46:14
maximum(d2fvpavperp_dvpa2_err): 0.037020262846521934
maximum(d2fvpavperp_dvperp2_err): 0.3770260641073364
begin C calculation   15:46:14
finished C calculation   15:46:14
H boundary data:
max(lower_vpa_err) = 5.033093830597579e-8
max(upper_vpa_err) = 6.119434911577848e-8
max(upper_vperp_err) = 5.910123337948825e-8
dHdvpa boundary data:
max(lower_vpa_err) = 8.591454350032057e-9
max(upper_vpa_err) = 1.44962392506498e-8
max(upper_vperp_err) = 4.373966390469608e-9
made inputs
vpa: ngrid: 9 nelement: 8 Lvpa: 12.0
vperp: ngrid: 9 nelement: 4 Lvperp: 6.0
dHdvperp boundary data:
max(lower_vpa_err) = 2.8658493436608395e-7
max(upper_vpa_err) = 5.217521526993535e-7
max(upper_vperp_err) = 1.182020113198179e-6
G boundary data:
max(lower_vpa_err) = 2.8720539795301647e-6
max(upper_vpa_err) = 2.6316899885969747e-6
max(upper_vperp_err) = 2.8720539795301647e-6
dGdvperp boundary data:
max(lower_vpa_err) = 2.482903449796403e-5
max(upper_vpa_err) = 2.909735053069351e-5
max(upper_vperp_err) = 3.7915328409021676e-5
d2Gdvperp2 boundary data:
max(lower_vpa_err) = 5.220852175691348e-6
max(upper_vpa_err) = 6.833840388242862e-6
max(upper_vperp_err) = 2.559078259925518e-6
d2Gdvperpdvpa boundary data:
max(lower_vpa_err) = 1.970150386283731e-6
max(upper_vpa_err) = 2.5303099550344488e-6
max(upper_vperp_err) = 2.3735468488647604e-6
d2Gdvpa2 boundary data:
max(lower_vpa_err) = 1.57749113285921e-8
max(upper_vpa_err) = 4.5085522318993876e-8
max(upper_vperp_err) = 6.921543632687488e-8
maximum(H_M_err): 4.107103661143974e-5 L2(H_M_err): 1.7569120964195357e-6
maximum(dHdvpa_M_err): 0.0004955837993653733 L2(dHdvpa_M_err): 2.276993160509552e-5
maximum(dHdvperp_M_err): 0.001746008530774383 L2(dHdvperp_M_err): 2.4832671932348726e-5
maximum(G_M_err): 7.047440409468919e-6 L2(G_M_err): 1.7116782251493884e-6
maximum(d2Gdvpa2_M_err): 4.3079008087976156e-5 L2(d2Gdvpa2_M_err): 1.8854973447370935e-6
maximum(dGdvperp_M_err): 0.00011765701814667184 L2(dGdvperp_M_err): 2.3875209521925354e-5
maximum(d2Gdvperpdvpa_M_err): 9.722524320501802e-5 L2(d2Gdvperpdvpa_M_err): 4.959196487603135e-6
maximum(d2Gdvperp2_M_err): 0.00011531799614450389 L2(d2Gdvperp2_M_err): 5.213248984043944e-6
maximum(C_M_err): 0.05721525593459045 L2(C_M_err): 0.0008978729183631709
dSdt: 1.3514867810199284e-5 should be >0.0
delta_n: -6.488385406775436e-14
delta_upar: 1.3719370946109822e-7
delta_pressure: 3.441977966961419e-6
made inputs
vpa: ngrid: 9 nelement: 8 Lvpa: 12.0
vperp: ngrid: 9 nelement: 4 Lvperp: 6.0
setting up GL quadrature   15:46:16
beginning (boundary) weights calculation   15:46:16
finished (boundary) weights calculation   15:46:26
begin elliptic operator assignment   15:46:26
finished elliptic operator constructor assignment   15:46:26
finished LU decomposition initialisation   15:46:26
finished YY array calculation   15:46:26
maximum(d2fvpavperp_dvpa2_err): 0.0005412865497824715
maximum(d2fvpavperp_dvperp2_err): 0.001115289035870326
begin C calculation   15:46:26
finished C calculation   15:46:26
H boundary data:
max(lower_vpa_err) = 2.391142839286431e-13
max(upper_vpa_err) = 1.0134393324534585e-12
max(upper_vperp_err) = 1.511013536514838e-13
dHdvpa boundary data:
max(lower_vpa_err) = 7.147823999353875e-13
max(upper_vpa_err) = 1.2617976108408868e-11
max(upper_vperp_err) = 1.3099243911796066e-12
dHdvperp boundary data:made inputs
vpa: ngrid: 9 nelement: 16 Lvpa: 12.0
vperp: ngrid: 9 nelement: 8 Lvperp: 
max(lower_vpa_err) = 1.3158751397540636e-9
6.0
max(upper_vpa_err) = 2.5699080745400504e-9
max(upper_vperp_err) = 4.6776645820645335e-9
G boundary data:
max(lower_vpa_err) = 8.236078485879261e-12
max(upper_vpa_err) = 7.192468842731614e-12
max(upper_vperp_err) = 6.6204819404447335e-12
dGdvperp boundary data:
max(lower_vpa_err) = 1.0859945209773514e-7
max(upper_vpa_err) = 1.2787998415753066e-7
max(upper_vperp_err) = 1.655506264297557e-7
d2Gdvperp2 boundary data:
max(lower_vpa_err) = 2.3702817791848574e-8
max(upper_vpa_err) = 3.273023418959298e-8
max(upper_vperp_err) = 1.0551617302123617e-8
d2Gdvperpdvpa boundary data:
max(lower_vpa_err) = 9.026868641459629e-9
max(upper_vpa_err) = 1.2333493323857248e-8
max(upper_vperp_err) = 1.0473630775165166e-8
d2Gdvpa2 boundary data:
max(lower_vpa_err) = 7.919269840589438e-12
max(upper_vpa_err) = 1.8862446327094773e-12
max(upper_vperp_err) = 6.89148044186183e-12
maximum(H_M_err): 1.0258306626376168e-7 L2(H_M_err): 4.140363342472842e-9
maximum(dHdvpa_M_err): 1.037256485453275e-6 L2(dHdvpa_M_err): 3.8851449806275776e-8
maximum(dHdvperp_M_err): 1.5149074440442417e-5 L2(dHdvperp_M_err): 9.042747201340302e-8
maximum(G_M_err): 1.4618293908696955e-8 L2(G_M_err): 6.131012938994507e-10
maximum(d2Gdvpa2_M_err): 1.0014428519777852e-7 L2(d2Gdvpa2_M_err): 3.92204018954867e-9
maximum(dGdvperp_M_err): 1.6795388385164312e-6 L2(dGdvperp_M_err): 1.0456903156843106e-7
maximum(d2Gdvperpdvpa_M_err): 1.1786526828020186e-6 L2(d2Gdvperpdvpa_M_err): 1.2348872969499483e-8
maximum(d2Gdvperp2_M_err): 2.59498948440573e-7 L2(d2Gdvperp2_M_err): 1.0488300846789792e-8
maximum(C_M_err): 0.0004990435648009392 L2(C_M_err): 6.910591096592381e-6
dSdt: -2.385963868838533e-9 should be >0.0
delta_n: -3.712863441841582e-14
delta_upar: -2.9382716212787143e-10
delta_pressure: -1.0184583000466452e-9
made inputs
vpa: ngrid: 9 nelement: 16 Lvpa: 12.0
vperp: ngrid: 9 nelement: 8 Lvperp: 6.0
setting up GL quadrature   15:46:28
beginning (boundary) weights calculation   15:46:28
finished (boundary) weights calculation   15:47:33
begin elliptic operator assignment   15:47:33
finished elliptic operator constructor assignment   15:47:33
finished LU decomposition initialisation   15:47:35
finished YY array calculation   15:47:35
maximum(d2fvpavperp_dvpa2_err): 7.316366210652347e-7
maximum(d2fvpavperp_dvperp2_err): 0.0002064141369515493
begin C calculation   15:47:35
finished C calculation   15:47:36
H boundary data:
max(lower_vpa_err) = 6.55586696041155e-14
max(upper_vpa_err) = 1.2359280265883399e-12
max(upper_vperp_err) = 1.3050671654468715e-13
dHdvpa boundary data:
max(lower_vpa_err) = 6.455357082213453e-13
max(upper_vpa_err) = 1.2566801765867552e-11
max(upper_vperp_err) = 1.3149255989608477e-12
dHdvperp boundary data:
max(lower_vpa_err) = 3.3394832038569078e-12
max(upper_vpa_err) = 6.728595111638036e-12
max(upper_vperp_err) = 1.1441802366674025e-11made inputs
G boundary data:
max(lower_vpa_err) = 
1.0482281709300878e-11
max(upper_vpa_err) = vpa: ngrid: 4.670930309202959e-12
max(upper_vperp_err) = 1.0482281709300878e-11
9 nelement: 32 Lvpa: 12.0dGdvperp boundary data:
max(lower_vpa_err) = 
vperp: ngrid: 2.7207158748154825e-10
max(upper_vpa_err) = 93.210289811761413e-10
 nelement: 16 Lvperp: max(upper_vperp_err) = 4.1453440680072617e-106.0
d2Gdvperp2 boundary data:

max(lower_vpa_err) = 5.971750871580639e-11
max(upper_vpa_err) = 8.463768574884512e-11
max(upper_vperp_err) = 2.6020560706108142e-11
d2Gdvperpdvpa boundary data:
max(lower_vpa_err) = 2.005917654201994e-11
max(upper_vpa_err) = 3.957241478946827e-11
max(upper_vperp_err) = 3.395795450389372e-11
d2Gdvpa2 boundary data:
max(lower_vpa_err) = 7.808765353084501e-12
max(upper_vpa_err) = 1.1249438880422957e-12
max(upper_vperp_err) = 7.005999946851915e-12
maximum(H_M_err): 2.1565749186436278e-10 L2(H_M_err): 1.0900550685061967e-11
maximum(dHdvpa_M_err): 7.508234312059159e-10 L2(dHdvpa_M_err): 2.8087026957853987e-11
maximum(dHdvperp_M_err): 3.1526424752043036e-9 L2(dHdvperp_M_err): 5.086395441354757e-11
maximum(G_M_err): 2.880318206166521e-11 L2(G_M_err): 6.346685211114848e-12
maximum(d2Gdvpa2_M_err): 1.808784233503502e-10 L2(d2Gdvpa2_M_err): 9.285752804634746e-12
maximum(dGdvperp_M_err): 4.6466860394167586e-10 L2(dGdvperp_M_err): 2.608152442797641e-10
maximum(d2Gdvperpdvpa_M_err): 2.5407111440764463e-10 L2(d2Gdvperpdvpa_M_err): 1.7735397828036034e-11
maximum(d2Gdvperp2_M_err): 2.0897625852001056e-9 L2(d2Gdvperp2_M_err): 2.543137535365888e-11
maximum(C_M_err): 5.9210482179722876e-6 L2(C_M_err): 4.732846507554791e-8
dSdt: 1.335817720875006e-12 should be >0.0
delta_n: -5.0655872581555606e-14
delta_upar: 6.57699912172338e-12
delta_pressure: 3.1181350134468236e-12
made inputs
vpa: ngrid: 9 nelement: 32 Lvpa: 12.0
vperp: ngrid: 9 nelement: 16 Lvperp: 6.0
setting up GL quadrature   15:47:37
beginning (boundary) weights calculation   15:47:37
finished (boundary) weights calculation   15:55:37
begin elliptic operator assignment   15:55:37
finished elliptic operator constructor assignment   15:55:39
finished LU decomposition initialisation   15:55:46
finished YY array calculation   15:55:46
maximum(d2fvpavperp_dvpa2_err): 8.371503046333828e-9
maximum(d2fvpavperp_dvperp2_err): 1.2967099038974084e-6
begin C calculation   15:55:46
finished C calculation   15:55:48
H boundary data:
max(lower_vpa_err) = 7.077671781985373e-14
max(upper_vpa_err) = 1.243172231824019e-12
max(upper_vperp_err) = 1.3547496457988473e-13
dHdvpa boundary data:
max(lower_vpa_err) = 6.453171330633722e-13
max(upper_vpa_err) = 1.256716952724446e-11
max(upper_vperp_err) = 1.3147295752080623e-12
dHdvperp boundary data:
max(lower_vpa_err) = 6.81839533564401e-14
max(upper_vpa_err) = 5.839096567372692e-13
max(upper_vperp_err) = 2.1690982343613996e-14
G boundary data:
max(lower_vpa_err) = 1.0789591442517121e-11
max(upper_vpa_err) = 4.82192064055198e-12
max(upper_vperp_err) = 1.0802025940392923e-11
dGdvperp boundary data:
max(lower_vpa_err) = 1.2048094726740954e-12
max(upper_vpa_err) = 7.611689056830073e-13
max(upper_vperp_err) = 7.58948459633757e-13
d2Gdvperp2 boundary data:
max(lower_vpa_err) = 1.792676562750728e-10
max(upper_vpa_err) = 6.650288653098357e-11
max(upper_vperp_err) = 5.0112691774018003e-14
d2Gdvperpdvpa boundary data:
max(lower_vpa_err) = 3.4659983216833723e-12
max(upper_vpa_err) = 7.7760090033685e-12
max(upper_vperp_err) = 7.7760090033685e-12
d2Gdvpa2 boundary data:
max(lower_vpa_err) = 7.808613998461222e-12
max(upper_vpa_err) = 1.1251190951133694e-12
max(upper_vperp_err) = 7.0055974910054886e-12
maximum(H_M_err): 1.243172231824019e-12 L2(H_M_err): 1.0311226876162984e-13
maximum(dHdvpa_M_err): 1.256716952724446e-11 L2(dHdvpa_M_err): 1.4167650084875205e-12
maximum(dHdvperp_M_err): 6.490801125746959e-11 L2(dHdvperp_M_err): 1.6199107424613495e-13
maximum(G_M_err): 1.0802025940392923e-11 L2(G_M_err): 6.014689955128148e-12
maximum(d2Gdvpa2_M_err): 7.808613998461222e-12 L2(d2Gdvpa2_M_err): 5.057976700506263e-12
maximum(dGdvperp_M_err): 7.858055352250037e-12 L2(dGdvperp_M_err): 3.5404500485514694e-13
maximum(d2Gdvperpdvpa_M_err): 7.7760090033685e-12 L2(d2Gdvperpdvpa_M_err): 4.3838690255226726e-12
maximum(d2Gdvperp2_M_err): 1.792676562750728e-10 L2(d2Gdvperp2_M_err): 1.1762046331907746e-12
maximum(C_M_err): 1.324478287841896e-8 L2(C_M_err): 1.9527409616988416e-10
dSdt: 1.3634826128537438e-13 should be >0.0
delta_n: -1.0422538990790483e-13
delta_upar: 5.7017280105170435e-12
delta_pressure: 9.624287673103972e-14
fkpl_C_G_H_max_test_ngrid_9_GLL.pdf
[[0.05721525593459045, 0.0004990435648009392, 5.9210482179722876e-6, 1.324478287841896e-8], [4.107103661143974e-5, 1.0258306626376168e-7, 2.1565749186436278e-10, 1.243172231824019e-12], [7.047440409468919e-6, 1.4618293908696955e-8, 2.880318206166521e-11, 1.0802025940392923e-11], [0.00390625, 1.52587890625e-5, 5.960464477539063e-8, 2.3283064365386963e-10], [0.0009765625, 9.5367431640625e-7, 9.313225746154785e-10, 9.094947017729282e-13]]
fkpl_coeffs_max_test_ngrid_9_GLL.pdf
[[0.0004955837993653733, 1.037256485453275e-6, 7.508234312059159e-10, 1.256716952724446e-11], [0.001746008530774383, 1.5149074440442417e-5, 3.1526424752043036e-9, 6.490801125746959e-11], [0.00011531799614450389, 2.59498948440573e-7, 2.0897625852001056e-9, 1.792676562750728e-10], [4.3079008087976156e-5, 1.0014428519777852e-7, 1.808784233503502e-10, 7.808613998461222e-12], [9.722524320501802e-5, 1.1786526828020186e-6, 2.5407111440764463e-10, 7.7760090033685e-12], [0.00011765701814667184, 1.6795388385164312e-6, 4.6466860394167586e-10, 7.858055352250037e-12], [0.00390625, 1.52587890625e-5, 5.960464477539063e-8, 2.3283064365386963e-10], [0.0009765625, 9.5367431640625e-7, 9.313225746154785e-10, 9.094947017729282e-13]]
fkpl_C_G_H_L2_test_ngrid_9_GLL.pdf
[[0.0008978729183631709, 6.910591096592381e-6, 4.732846507554791e-8, 1.9527409616988416e-10], [1.7569120964195357e-6, 4.140363342472842e-9, 1.0900550685061967e-11, 1.0311226876162984e-13], [1.7116782251493884e-6, 6.131012938994507e-10, 6.346685211114848e-12, 6.014689955128148e-12], [0.00390625, 1.52587890625e-5, 5.960464477539063e-8, 2.3283064365386963e-10], [0.0009765625, 9.5367431640625e-7, 9.313225746154785e-10, 9.094947017729282e-13]]
fkpl_coeffs_L2_test_ngrid_9_GLL.pdf
[[2.276993160509552e-5, 3.8851449806275776e-8, 2.8087026957853987e-11, 1.4167650084875205e-12], [2.4832671932348726e-5, 9.042747201340302e-8, 5.086395441354757e-11, 1.6199107424613495e-13], [5.213248984043944e-6, 1.0488300846789792e-8, 2.543137535365888e-11, 1.1762046331907746e-12], [1.8854973447370935e-6, 3.92204018954867e-9, 9.285752804634746e-12, 5.057976700506263e-12], [4.959196487603135e-6, 1.2348872969499483e-8, 1.7735397828036034e-11, 4.3838690255226726e-12], [2.3875209521925354e-5, 1.0456903156843106e-7, 2.608152442797641e-10, 3.5404500485514694e-13], [0.00390625, 1.52587890625e-5, 5.960464477539063e-8, 2.3283064365386963e-10], [0.0009765625, 9.5367431640625e-7, 9.313225746154785e-10, 9.094947017729282e-13]]
fkpl_conservation_test_ngrid_9_GLL.pdf
[[0.05721525593459045, 0.0004990435648009392, 5.9210482179722876e-6, 1.324478287841896e-8], [0.0008978729183631709, 6.910591096592381e-6, 4.732846507554791e-8, 1.9527409616988416e-10], [6.488385406775436e-14, 3.712863441841582e-14, 5.0655872581555606e-14, 1.0422538990790483e-13], [1.3719370946109822e-7, 2.9382716212787143e-10, 6.57699912172338e-12, 5.7017280105170435e-12], [3.441977966961419e-6, 1.0184583000466452e-9, 3.1181350134468236e-12, 9.624287673103972e-14], [0.00390625, 1.52587890625e-5, 5.960464477539063e-8, 2.3283064365386963e-10], [0.0009765625, 9.5367431640625e-7, 9.313225746154785e-10, 9.094947017729282e-13]]
fkpl_timing_test_ngrid_9_GLL.pdf
[[129.0, 185.0, 390.0, 1731.0], [2406.0, 10645.0, 66885.0, 488427.0], [4.0, 16.0, 64.0, 256.0], [8.0, 64.0, 512.0, 4096.0]]

and

$ mpirun -np 2 bin/julia --project -O3 --check-bounds=no test_scripts/fkpl_direct_integration_test.jl 
  Activating project at `~/physics/moment_kinetics-master`
  Activating project at `~/physics/moment_kinetics-master`
beginning allocation   15:35:56
setting up input arrays   15:35:56
finished integration   15:35:57
max_dfsdvpa_err: 0.06917634554642549
max_dfsdvperp_err: 0.007822928466986005
max_d2fsdvperpdvpa_err: 0.04229258103363549
max_GG_err: 0.1727092294104633 CartesianIndex(1, 9)
spot check GG_err: 0.17270922941046152 GG: 8.77584173384679
max_HH_err: 0.01176820680772439 CartesianIndex(7, 1)
spot check HH_err: 0.0023623267529214176 HH: 0.12021345695067934
max_dHdvperp_err: 0.005060999137775052 CartesianIndex(5, 2)
spot check dHdvperp_err: 0.00020813131759352468 dHdvperp: -0.010029058834073337
max_dHdvpa_err: 0.03378389707838325 CartesianIndex(6, 1)
spot check dHdvpa_err: 0.00019892236111920467 dHdvpa: -0.010019849877599017
max_d2Gdvperp2_err: 0.010012878555139604 CartesianIndex(6, 1)
spot check d2Gdvperp2_err: 0.0014716888282249319 d2Gdvperp2: 0.06121566455347721
max_d2Gdvperpdvpa_err: 0.008075061155560184 CartesianIndex(7, 3)
spot check d2Gdvperpdpva_err: 0.0013191685413440693 d2Gdvperpdvpa: -0.05778950176110308
max_dGdvperp_err: 0.022655593570891863 CartesianIndex(5, 9)
spot check dGdvperp_err: 0.016057715180843646 dGdvperp: 0.7133435688509114
max_d2Gdvpa2_err: 0.023991813982288446 CartesianIndex(5, 1)
spot check d2Gdvpa2_err: 0.0010498902669867638 d2Gdvpa2: 0.06079386599223904
beginning allocation   15:35:57
setting up input arrays   15:35:57
finished integration   15:35:59
max_dfsdvpa_err: 0.007526978106374075
max_dfsdvperp_err: 0.006522786073746775
max_d2fsdvperpdvpa_err: 0.004438639792646248
max_GG_err: 0.002114116752125028 CartesianIndex(1, 17)
spot check GG_err: 0.0021141167521214754 GG: 8.601018387684206
max_HH_err: 0.00041574138745459344 CartesianIndex(9, 1)
spot check HH_err: 2.9279087745456844e-5 HH: 0.11782185111001246
max_dHdvperp_err: 0.0002472967782334301 CartesianIndex(9, 4)
spot check dHdvperp_err: 3.393611334105706e-6 dHdvperp: -0.009817533905145706
max_dHdvpa_err: 0.001218738220830351 CartesianIndex(8, 1)
spot check dHdvpa_err: 2.6347196423488067e-6 dHdvpa: -0.009818292796837463
max_d2Gdvperp2_err: 0.00022961357643574987 CartesianIndex(11, 1)
spot check d2Gdvperp2_err: 2.141250911029402e-5 d2Gdvperp2: 0.059722563216141986
max_d2Gdvperpdvpa_err: 0.0001280348097989839 CartesianIndex(8, 4)
spot check d2Gdvperpdpva_err: 2.110720480120315e-5 d2Gdvperpdvpa: -0.05644922601495781
max_dGdvperp_err: 0.00034165845479283696 CartesianIndex(9, 17)
spot check dGdvperp_err: 0.0002467831914247931 dGdvperp: 0.697039070478643
max_d2Gdvpa2_err: 0.0009255003695752828 CartesianIndex(9, 1)
spot check d2Gdvpa2_err: 1.4088019676672392e-5 d2Gdvpa2: 0.05972988770557561
beginning allocation   15:35:59
setting up input arrays   15:35:59
finished integration   15:36:15
max_dfsdvpa_err: 0.00231656266399774
max_dfsdvperp_err: 0.00036035008063739667
max_d2fsdvperpdvpa_err: 0.0014054518308214345
max_GG_err: 5.9437303034926e-6 CartesianIndex(16, 1)
spot check GG_err: 3.526033951928298e-6 GG: 8.603128978402376
max_HH_err: 3.651098599011249e-5 CartesianIndex(15, 4)
spot check HH_err: 4.6574566064938594e-8 HH: 0.11785108362319185
max_dHdvperp_err: 2.731566442623623e-5 CartesianIndex(19, 4)
spot check dHdvperp_err: 3.6741582484789426e-9 dHdvperp: -0.009820923842321564
max_dHdvpa_err: 0.00014805273053600243 CartesianIndex(19, 4)
spot check dHdvpa_err: 4.895134292706715e-9 dHdvpa: -0.00982092262134552
max_d2Gdvperp2_err: 4.462995049858698e-6 CartesianIndex(15, 4)
spot check d2Gdvperp2_err: 5.381801379544804e-8 d2Gdvperp2: 0.059743921907238484
max_d2Gdvperpdvpa_err: 8.90463608107378e-6 CartesianIndex(15, 7)
spot check d2Gdvperpdpva_err: 3.9038413043868125e-8 d2Gdvperpdvpa: -0.05647029418134597
max_dGdvperp_err: 2.635666952466309e-6 CartesianIndex(18, 8)
spot check dGdvperp_err: 4.338884896926132e-7 dGdvperp: 0.6972854197815781
max_d2Gdvpa2_err: 2.785981831199269e-5 CartesianIndex(16, 1)
spot check d2Gdvpa2_err: 1.764256749720028e-8 d2Gdvpa2: 0.05974395808268478
beginning allocation   15:36:15
setting up input arrays   15:36:15
finished integration   15:39:36
max_dfsdvpa_err: 0.00026575414003554565
max_dfsdvperp_err: 2.1172776407757388e-5
max_d2fsdvperpdvpa_err: 0.0001642877745444693
max_GG_err: 1.7433513477271845e-7 CartesianIndex(65, 65)
spot check GG_err: 1.7433513477271845e-7 GG: 8.603132330101193
max_HH_err: 1.3506447764988216e-5 CartesianIndex(35, 4)
spot check HH_err: 1.870354898736437e-9 HH: 0.11785112832740302
max_dHdvperp_err: 5.269925131543007e-6 CartesianIndex(35, 4)
spot check dHdvperp_err: 2.0130052104383456e-9 dHdvperp: -0.009820925503474602
max_dHdvpa_err: 9.697540046677e-6 CartesianIndex(39, 4)
spot check dHdvpa_err: 7.243554089875204e-10 dHdvpa: -0.009820926792124403
max_d2Gdvperp2_err: 4.092511999287396e-7 CartesianIndex(31, 4)
spot check d2Gdvperp2_err: 2.0419232582558067e-8 d2Gdvperp2: 0.0597439553060197
max_d2Gdvperpdvpa_err: 4.144452885046501e-8 CartesianIndex(35, 12)
spot check d2Gdvperpdpva_err: 1.4728007056985515e-8 d2Gdvperpdvpa: -0.05647031849175196
max_dGdvperp_err: 2.325342763898064e-7 CartesianIndex(33, 65)
spot check dGdvperp_err: 1.7169500243774394e-7 dGdvperp: 0.6972856819750654
max_d2Gdvpa2_err: 3.5191280134405645e-7 CartesianIndex(36, 4)
spot check d2Gdvpa2_err: 6.389314924359901e-9 d2Gdvpa2: 0.059743969335937355
beginning allocation   15:39:36
setting up input arrays   15:39:36
finished integration   16:14:49
max_dfsdvpa_err: 2.1171562811284073e-5
max_dfsdvperp_err: 1.4593235959925455e-6
max_d2fsdvperpdvpa_err: 1.3403331560288279e-5
max_GG_err: 1.7445086442080537e-7 CartesianIndex(1, 129)
spot check GG_err: 1.7445085020995066e-7 GG: 8.603132329985478
max_HH_err: 3.770092023014371e-6 CartesianIndex(63, 4)
spot check HH_err: 1.8718619154700633e-9 HH: 0.117851128325896
max_dHdvperp_err: 1.119447272046603e-6 CartesianIndex(67, 20)
spot check dHdvperp_err: 2.071951946819306e-9 dHdvperp: -0.009820925444527865
max_dHdvpa_err: 2.3485358356389874e-6 CartesianIndex(75, 4)
spot check dHdvpa_err: 7.243466260825615e-10 dHdvpa: -0.009820926792133186
max_d2Gdvperp2_err: 9.223546071357447e-8 CartesianIndex(67, 20)
spot check d2Gdvperp2_err: 2.0755675851635846e-8 d2Gdvperp2: 0.05974395496957643
max_d2Gdvperpdvpa_err: 1.5532910201121464e-8 CartesianIndex(114, 129)
spot check d2Gdvperpdpva_err: 1.506689992908372e-8 d2Gdvperpdvpa: -0.056470318152859085
max_dGdvperp_err: 2.3828071060716383e-7 CartesianIndex(65, 129)
spot check dGdvperp_err: 1.7583306866075787e-7 dGdvperp: 0.6972856778369991
max_d2Gdvpa2_err: 3.901118794669145e-8 CartesianIndex(56, 4)
spot check d2Gdvpa2_err: 6.389800050188299e-9 d2Gdvpa2: 0.05974396933545223
fkpl_essential_coeffs_numerical_lagrange_integration_test_ngrid_5_GLL.pdf
fkpl_potentials_numerical_lagrange_integration_test_ngrid_5_GLL.pdf
fkpl_fs_numerical_test_ngrid_5_GLL.pdf
mrhardman commented 3 months ago

@johnomotani I am now happy that I understand the main points of this PR, so just making a link to the any documentation https://mabarnes.github.io/moment_kinetics/previews/PR176/developing/#Collision-operator-and-anyv-region in the appropriate place mentioned in one of our conversations above should be the last thing that needs addressing. Thank you for getting my test scripts up to date with the source. Please feel free to merge this in the master when you are ready!