Currently failing longtests and examples

[tmbgreaves]

This issue documents the remaining failing tests following the Actions/GCC10 PR work. I'll update this initial post with the latest details - feel free to edit if you have updates, or note in the discussion below.

There are currently six failing tests/examples needing attention:

• EXAMPLE: driven_cavity : One failure, see note below
• EXAMPLE: lock_exchange : 'TypeError: only integer scalar arrays can be converted to a scalar index' [0]
• LONGTEST: lock_exchange_3d_parallel : 'Error: Process completed with exit code 137.' [1]
• LONGTEST: saltfinger2d_adaptive : GMSH erroring during mesh generation
• VLONGTEST: backward_facing_step_2d_longtest : Two testing failures, see below.
• VLONGTEST: flredecomp_long : timing out after three days, needs rerunning manually.

VLong tests are running on the longtests repo at https://github.com/FluidityProject/longtests/actions

[0] https://github.com/FluidityProject/fluidity/runs/2522255272?check_suite_focus=true [1] https://github.com/FluidityProject/fluidity/runs/2522625135?check_suite_focus=true

[tmbgreaves]

Question moved from PR #311

@angus-g - I think I have a question for you: it looks like some of the examples / longtests which parse detectors are failing as a result of detector files now outputting as h5part, but the examples/tests are still configured to read old-style detectors output via stat_parse from the fluidity_tools module. On a first pass, tests which I suspect are failing from this are:

• wetting_and_drying_balzano1_dg_parallel
• hokkaido-nansei-oki_tsunami
• lock_exchange_2d_Lagrangian_paths
• square-convection-1e6
• Stokes_subduction_zone_vanKeken2008_OneA

Does this sound like it could be a reasonable diagnosis and if so how straightforward would a fix be?

[tmbgreaves]

Question moved from PR #311

UPDATED 2021/05/06 @jhill1 - I think I've now fixed the medp1dgp2 utScan bug.

[tmbgreaves]

Question moved from PR #311

@matt-piggott - I'm seeing what I hope is a marginal test failure in the driven_cavity example on Ubuntu 20.04, where a test of convergence > 1.85 fails on:

driven_cavity: Running Quantity4ConvergesToSecondOrder: [[1.72761663] [2.23112378] [2.61936127]] driven_cavity: failure.

Can this be relaxed slightly?

[tmbgreaves]

@drhodrid - a follow-up on the updated particle tests, I don't think I've messed up the merge of your updates but am seeing what looks like a significant RMS error change from particle_entrainment_of_dense_layer, see:

https://github.com/FluidityProject/fluidity/runs/2509562204?check_suite_focus=true

Test is '560 +- 10', actual value is 90.25800673280742

[tmbgreaves]

@drhodrid - in particle_rayleigh_taylor_mu10fold on Focal I'm seeing one failure of 'Max entrainment less than 0.9' but a test value of 'Max Entrainment 0.903234453893775'. Am I OK to relax that slightly to get a pass?

[tmbgreaves]

Documenting BFS2D longtest failures:

Test: Running reference reattachment length is about 14.0 Value: Assigning reference_reattachment_length = [[ 14.60728793 60.00759976] [ 14.46771101 70.00617095] [ 14.34311088 80.00749879] [ 13.94609803 90.00016529] [ 14.14875746 100.01463062] [ 14.11128432 110.01588166] [ 14.30831721 120.0105066 ] [ 13.11879199 130.01656148] [ 13.82287353 140.01110268] [ 13.5161791 150.00253021] [ 13.19420174 160.01205175] [ 13.09551441 170.0267938 ] [ 12.79628411 180.01357642] [ 12.66541669 190.00075329] [ 12.54315814 200.0169508 ]]

Test: Running kepsilon reattachment length is approximately equal to the experimental value of 7.0 Value: Assigning kepsilon_reattachment_length = [[ 5.94598516 130.03556546] [ 5.94630335 140.03053697] [ 5.94655816 150.02550848] [ 5.94676675 160.02047999] [ 5.94694063 170.0154515 ] [ 5.94708778 180.010423 ] [ 5.94721393 190.00539451] [ 5.94732327 200.00036601]]

Completed run at : https://github.com/FluidityProject/longtests/runs/2527092698?check_suite_focus=true

[drhodrid]

Hi Tim. I just pushed some changes that will hopefully fix particle_entrainment_of_dense_layer. Please let me know if that's the case when its run through.

[tmbgreaves]

Thanks Rhod, that's running now. I'll post results as they come in.

Update: test has run for significantly longer this time, which looks promising.

[drhodrid]

Ahhh damn! Back on my list for next week then!! R

On 14 May 2021, at 8:17 pm, Tim Greaves @.**@.>> wrote:

@drhodridhttps://github.com/drhodrid - I'm still seeing the Max RMS error at ~90, giving a failure:

https://github.com/FluidityProject/fluidity/runs/2582995968?check_suite_focus=true

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://github.com/FluidityProject/fluidity/issues/316#issuecomment-841153262, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AB25UKXJZXOORL7TM3XZNKDTNT2CRANCNFSM44FTYVJA.

[tmbgreaves]

@drhodrid - this time was my error I think - sorry! I was running the previous version by mistake. But it still needs testing as it overran the 6h time limit on the standard queue, so I'm rerunning on the vlong queue now and will update with results.

[drhodrid]

Thanks Tim. If it passes, I can probably improve efficiency somewhat to get it under 6 hours.

R

On 17 May 2021, at 6:41 pm, Tim Greaves @.**@.>> wrote:

@drhodridhttps://github.com/drhodrid - this time was my error I think - sorry! I was running the previous version by mistake. But it still needs testing as it overran the 6h time limit on the standard queue, so I'm rerunning on the vlong queue now and will update with results.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://github.com/FluidityProject/fluidity/issues/316#issuecomment-842138346, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AB25UKSVTRHI3QINGB5XELTTODJCLANCNFSM44FTYVJA.

[tmbgreaves]

Run started at https://github.com/FluidityProject/longtests/actions/runs/849706302

[tmbgreaves]

@drhodrid - confirming pass: https://github.com/FluidityProject/longtests/actions/runs/849706302

Could I also nudge you on the hopefully-tolerance failure in particle_rayleigh_taylor_mu10fold ? I'm seeing one failure of 'Max entrainment less than 0.9' but a test value of 'Max Entrainment 0.903234453893775'. Am I OK to relax that slightly to get a pass?

[drhodrid]

Hi Tim,

Great - yes please relax tolerance for that case to 0.91.

Best wishes,

Rhod

On 18 May 2021, at 5:52 pm, Tim Greaves @.**@.>> wrote:

@drhodridhttps://github.com/drhodrid - confirming pass: https://github.com/FluidityProject/longtests/actions/runs/849706302

Could I also nudge you on the hopefully-tolerance failure in particle_rayleigh_taylor_mu10fold ? I'm seeing one failure of 'Max entrainment less than 0.9' but a test value of 'Max Entrainment 0.903234453893775'. Am I OK to relax that slightly to get a pass?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://github.com/FluidityProject/fluidity/issues/316#issuecomment-842943330, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AB25UKROWFWGMIIOTUCY2ATTOIMFLANCNFSM44FTYVJA.

[tmbgreaves]

Thanks @drhodrid - done and re-running at https://github.com/FluidityProject/longtests/actions/runs/852603097

[tmbgreaves]

@drhodrid - another of yours, I'm afraid - Stokes_mms_p1dg_p2 is failing to converge. Might you be able to take a look when convenient and see if it's still a test you want, and if so propose a fix?

[drhodrid]

Hi Tim,

Fixed a few issues with this test today. Please let me know if they were sufficient!

R

On 18 May 2021, at 10:43 pm, Tim Greaves @.**@.>> wrote:

@drhodridhttps://github.com/drhodrid - another of yours, I'm afraid - Stokes_mms_p1dg_p2 is failing to converge. Might you be able to take a look when convenient and see if it's still a test you want, and if so propose a fix?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://github.com/FluidityProject/fluidity/issues/316#issuecomment-843138183, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AB25UKU2TLZWVIT2EBV4OADTOJOGDANCNFSM44FTYVJA.

[tmbgreaves]

Perfect, all passing now, thank you @drhodrid

[tmbgreaves]

@javalosp - this is the issue documenting failing tests :-) See the first entry for the summary of where we are at the moment - I think driven_cavity is (hopefully) a straightforward one to tackle first, as it looks like a tolerance failure.

[tmbgreaves]

Here's a method for reproducing the failures using a test environment generated by Actions:

docker run -it fluidity/actions:focal-68ec40081346d1dab836f23b92742315a9c72430 bash -il
./bin/testharness -f <test>.xml

[tmbgreaves]

Noting that @Patol75 is working on a general testing rewrite in PR #326 and has fixed at least some of these failures.