trixi-framework / Trixi.jl

Trixi.jl: Adaptive high-order numerical simulations of conservation laws in Julia
https://trixi-framework.github.io/Trixi.jl
MIT License
519 stars 100 forks source link

Failing elixirs on Apple Silicon #1838

Open efaulhaber opened 6 months ago

efaulhaber commented 6 months ago

As I mentioned in #1830, many elixirs are still not working on Apple Silicon on multiple threads. To determine a definite list of elixirs, I started running all tests on 2 threads (ignoring allocation tests, which will fail on multiple threads due to extra allocations).

Here is the summary for TreeMesh. I'll keep running more tests in the background. The failing tests are due to slight differences in the L2/Linf errors. The errors are all due to LoadError: cfunction: closures are not supported on this platform (see #1462 for a solution that has to be applied to the remaining indicators).

Test Summary:                                                                          | Pass  Fail  Error  Total     Time
TreeMesh1D                                                                             |  973     2      4    979  4m50.9s
  Examples 1D                                                                          |  900     2      4    906  4m37.0s
    Linear scalar advection                                                            |   21                  21     0.9s
    Inviscid Burgers                                                                   |   20                  20     8.1s
    Hyperbolic diffusion                                                               |   14                  14     1.8s
    Compressible Euler                                                                 |  186            1    187    25.1s
      elixir_euler_source_terms.jl                                                     |    9                   9     0.1s
      elixir_euler_convergence_pure_fv.jl                                              |    9                   9     0.1s
      elixir_euler_density_wave.jl                                                     |    9                   9     0.1s
      elixir_euler_density_wave.jl with initial_condition_constant                     |    9                   9     0.1s
      elixir_euler_source_terms_nonperiodic.jl                                         |    9                   9     0.1s
      elixir_euler_ec.jl                                                               |    9                   9     0.1s
      elixir_euler_ec.jl with flux_kennedy_gruber                                      |    9                   9     0.0s
      elixir_euler_ec.jl with flux_shima_etal                                          |    9                   9     0.0s
      elixir_euler_ec.jl with flux_chandrashekar                                       |    9                   9     0.1s
      elixir_euler_ec.jl with flux_hll                                                 |    9                   9     0.1s
      elixir_euler_shockcapturing.jl                                                   |    9                   9     0.1s
      elixir_euler_sedov_blast_wave.jl                                                 |    9                   9     2.8s
      elixir_euler_sedov_blast_wave.jl (HLLE)                                          |    9                   9     3.1s
      elixir_euler_sedov_blast_wave_pure_fv.jl                                         |    9                   9     2.6s
      elixir_euler_sedov_blast_wave.jl with pressure                                   |    9                   9     2.7s
      elixir_euler_sedov_blast_wave.jl with density                                    |    9                   9     2.6s
      elixir_euler_positivity.jl                                                       |                 1      1     2.5s
      elixir_euler_blast_wave.jl                                                       |    9                   9     1.8s
      elixir_euler_quasi_1d_source_terms.jl                                            |   11                  11     0.1s
      elixir_euler_quasi_1d_discontinuous.jl                                           |   11                  11     2.2s
      elixir_euler_quasi_1d_ec.jl                                                      |   11                  11     1.7s
    Compressible Euler Multicomponent                                                  |   80                  80     9.9s
    MHD                                                                                |  209                 209    53.6s
    MHD Multicomponent                                                                 |  113                 113    20.2s
    Compressible Euler with self-gravity                                               |    9                   9     6.4s
    Shallow Water                                                                      |  148     2      3    153  1m19.3s
      elixir_shallowwater_ec.jl                                                        |    9                   9     4.1s
      elixir_shallowwater_ec.jl with initial_condition_weak_blast_wave                 |    9                   9     2.3s
      elixir_shallowwater_well_balanced.jl                                             |    9                   9     2.4s
      elixir_shallowwater_well_balanced.jl with FluxHydrostaticReconstruction          |    9                   9     4.2s
      elixir_shallowwater_well_balanced.jl with flux_nonconservative_ersing_etal       |    9                   9     3.7s
      elixir_shallowwater_well_balanced_wet_dry.jl with FluxHydrostaticReconstruction  |                 1      1     6.1s
      elixir_shallowwater_source_terms.jl                                              |    9                   9     4.7s
      elixir_shallowwater_source_terms.jl with flux_hll                                |    9                   9     4.2s
      elixir_shallowwater_source_terms.jl with flux_nonconservative_ersing_etal        |    9                   9     4.0s
      elixir_shallowwater_source_terms_dirichlet.jl                                    |    9                   9     2.0s
      elixir_shallowwater_source_terms_dirichlet.jl with FluxHydrostaticReconstruction |    9                   9     4.3s
      elixir_shallowwater_well_balanced_nonperiodic.jl with Dirichlet boundary         |    9                   9     3.8s
      elixir_shallowwater_well_balanced_nonperiodic.jl with wall boundary              |    9                   9     2.5s
      elixir_shallowwater_shock_capturing.jl                                           |    9                   9     5.9s
      elixir_shallowwater_beach.jl                                                     |                 1      1     3.4s
      elixir_shallowwater_parabolic_bowl.jl                                            |                 1      1     5.8s
      elixir_shallow_water_quasi_1d_source_terms.jl                                    |   11                  11     4.4s
      elixir_shallowwater_quasi_1d_well_balanced.jl                                    |    9     2            11     4.8s
      elixir_shallowwater_quasi_1d_discontinuous.jl                                    |   11                  11     4.0s
    Shallow Water Two layer                                                            |   39                  39    13.1s
    Linear scalar advection                                                            |   10                  10    15.8s
    Inviscid Burgers                                                                   |   15                  15    17.2s
    Compressible Euler                                                                 |   36                  36    24.9s
  Tests for initial conditions                                                         |   20                  20     7.1s
  Displaying components 1D                                                             |   45                  45     3.1s
  Additional tests in 1D                                                               |    6                   6     0.0s
  Nonconservative terms in 1D (linear advection)                                       |    1                   1     3.6s
ERROR: LoadError: Some tests did not pass: 973 passed, 2 failed, 4 errored, 0 broken.
in expression starting at /Users/erik/git/Trixi.jl/test/test_tree_1d.jl:1

test_tree_2d_part1.jl passed.

Test Summary:                                                             | Pass  Fail  Error  Total     Time
TreeMesh2D Part 2                                                         |  592    14      3    609  7m32.4s
  Examples 2D                                                             |  591    14      3    608  7m32.3s
    Acoustic Perturbation                                                 |   85                  85    39.3s
    Linearized Euler Equations 2D                                         |   22                  22    19.8s
    Compressible Euler                                                    |  366     8      2    376  4m44.5s
      elixir_euler_source_terms.jl                                        |   11                  11     7.9s
      elixir_euler_convergence_pure_fv.jl                                 |   11                  11     9.1s
      elixir_euler_density_wave.jl                                        |   11                  11     7.4s
      elixir_euler_source_terms_nonperiodic.jl                            |   11                  11     3.9s
      elixir_euler_ec.jl                                                  |   11                  11     7.7s
      elixir_euler_ec.jl with flux_kennedy_gruber                         |   11                  11     7.9s
      elixir_euler_ec.jl with flux_chandrashekar                          |   11                  11     8.3s
      elixir_euler_shockcapturing.jl                                      |   11                  11    12.6s
      elixir_euler_shockcapturing_subcell.jl                              |   11                  11    16.6s
      elixir_euler_blast_wave.jl                                          |   11                  11    10.7s
      elixir_euler_blast_wave_pure_fv.jl                                  |   11                  11     5.2s
      elixir_euler_blast_wave_amr.jl                                      |   11                  11     7.0s
      elixir_euler_blast_wave_sc_subcell_nonperiodic.jl                   |   11                  11     3.4s
      elixir_euler_sedov_blast_wave.jl                                    |   11                  11    12.7s
      elixir_euler_sedov_blast_wave_sc_subcell.jl                         |    3     8            11    14.2s
      elixir_euler_sedov_blast_wave.jl (HLLE)                             |   11                  11    13.5s
      elixir_euler_positivity.jl                                          |                 1      1     6.0s
      elixir_euler_blob_mortar.jl                                         |   11                  11    10.6s
      elixir_euler_blob_amr.jl                                            |   11                  11    15.0s
      elixir_euler_kelvin_helmholtz_instability_fjordholm_etal.jl         |   11                  11    15.9s
      elixir_euler_kelvin_helmholtz_instability.jl                        |   11                  11     3.4s
      elixir_euler_kelvin_helmholtz_instability_amr.jl                    |   11                  11     5.2s
      elixir_euler_kelvin_helmholtz_instability_sc_subcell.jl             |   11                  11    18.2s
      elixir_euler_colliding_flow.jl                                      |   11                  11     4.8s
      elixir_euler_colliding_flow_amr.jl                                  |   11                  11     4.1s
      elixir_euler_astro_jet_amr.jl                                       |   11                  11     6.0s
      elixir_euler_vortex.jl                                              |   11                  11     4.2s
      elixir_euler_vortex_mortar.jl                                       |   11                  11     4.4s
      elixir_euler_vortex_mortar_split.jl                                 |   11                  11     7.8s
      elixir_euler_vortex_shockcapturing.jl                               |   11                  11     5.4s
      elixir_euler_vortex_mortar_shockcapturing.jl                        |   11                  11     3.1s
      elixir_euler_vortex_amr.jl                                          |   11                  11     6.9s
      elixir_euler_ec.jl with boundary_condition_slip_wall                |   11                  11     3.6s
      elixir_euler_warm_bubble.jl                                         |                 1      1     0.4s
      Compressible Euler: Tests for initial conditions                    |   22                  22     7.0s
    Compressible Euler Multicomponent                                     |  104     6           110  1m26.7s
      Testing entropy2cons and cons2entropy                               |    2                   2     0.5s
      elixir_eulermulti_shock_bubble.jl                                   |   13                  13    14.8s
      elixir_eulermulti_shock_bubble_shockcapturing_subcell_positivity.jl |   15                  15    15.5s
      elixir_eulermulti_shock_bubble_shockcapturing_subcell_minmax.jl     |    7     6            13     3.9s
      elixir_eulermulti_ec.jl                                             |   11                  11     9.0s
      elixir_eulermulti_es.jl                                             |   17                  17    11.6s
      elixir_eulermulti_convergence_ec.jl                                 |   13                  13    10.4s
      elixir_eulermulti_convergence_es.jl                                 |   13                  13     9.2s
      elixir_eulermulti_convergence_es.jl with flux_chandrashekar         |   13                  13    10.7s
    Polytropic Euler                                                      |    9                   9     9.2s
    Acoustic perturbation coupled with compressible Euler                 |                 1      1     1.0s
      elixir_euleracoustics_co-rotating_vortex_pair.jl                    |                 1      1     0.7s
    KPP                                                                   |    5                   5    11.4s
ERROR: LoadError: Some tests did not pass: 592 passed, 14 failed, 3 errored, 0 broken.
in expression starting at /Users/erik/git/Trixi.jl/test/test_tree_2d_part2.jl:1
Test Summary:                                                                         | Pass  Error  Total      Time
TreeMesh2D Part 3                                                                     |  648      3    651  10m34.6s
  Examples 2D                                                                         |  647      3    650  10m34.6s
    MHD                                                                               |  210           210   3m09.1s
    MHD Multicomponent                                                                |  117           117   1m40.1s
    Lattice-Boltzmann                                                                 |  105           105     49.9s
    Shallow Water                                                                     |  110      3    113   2m22.7s
      elixir_shallowwater_ec.jl                                                       |   11            11     13.2s
      elixir_shallowwater_well_balanced.jl                                            |   11            11      4.7s
      elixir_shallowwater_well_balanced_wall.jl                                       |   11            11      4.8s
      elixir_shallowwater_well_balanced.jl with FluxHydrostaticReconstruction         |   11            11     15.6s
      elixir_shallowwater_well_balanced.jl with flux_nonconservative_ersing_etal      |   11            11     11.7s
      elixir_shallowwater_well_balanced_wet_dry.jl with FluxHydrostaticReconstruction |           1      1     16.3s
      elixir_shallowwater_source_terms.jl                                             |   11            11     11.2s
      elixir_shallowwater_source_terms_dirichlet.jl                                   |   11            11      4.1s
      elixir_shallowwater_source_terms.jl with flux_hll                               |   11            11     11.6s
      elixir_shallowwater_source_terms.jl with flux_nonconservative_ersing_etal       |   11            11     10.6s
      elixir_shallowwater_conical_island.jl                                           |           1      1     19.8s
      elixir_shallowwater_parabolic_bowl.jl                                           |           1      1      4.2s
      elixir_shallowwater_wall.jl                                                     |   11            11     12.2s
    Two-Layer Shallow Water                                                           |   51            51     47.9s
    Linear scalar advection                                                           |   10            10     27.0s
    Compressible Euler                                                                |   44            44   1m16.9s
ERROR: LoadError: Some tests did not pass: 648 passed, 0 failed, 3 errored, 0 broken.
in expression starting at /Users/erik/git/Trixi.jl/test/test_tree_2d_part3.jl:1
Test Summary:                                                              | Pass  Error  Total      Time
TreeMesh3D Part 1                                                          |  222      1    223  10m08.4s
  Examples 3D                                                              |  221      1    222  10m08.3s
    Compressible Euler                                                     |  221      1    222  10m07.9s
      elixir_euler_source_terms.jl                                         |   13            13   1m09.7s
      elixir_euler_convergence_pure_fv.jl                                  |   13            13     30.9s
      elixir_euler_source_terms.jl with split_form                         |   13            13     28.1s
      elixir_euler_convergence.jl                                          |   13            13     30.7s
      elixir_euler_mortar.jl                                               |   13            13      2.8s
      elixir_euler_amr.jl                                                  |   13            13     36.2s
      elixir_euler_taylor_green_vortex.jl                                  |   13            13     29.2s
      elixir_euler_shockcapturing.jl                                       |   13            13     45.2s
      elixir_euler_shockcapturing_amr.jl                                   |   13            13      9.8s
      elixir_euler_density_pulse.jl                                        |   13            13     29.7s
      elixir_euler_ec.jl                                                   |   13            13     10.2s
      elixir_euler_ec.jl with initial_condition=initial_condition_constant |   13            13     11.2s
      elixir_euler_ec.jl with flux_chandrashekar                           |   13            13     27.7s
      elixir_euler_ec.jl with flux_kennedy_gruber                          |   13            13     32.0s
      elixir_euler_ec.jl with flux_shima_etal                              |   13            13     27.6s
      elixir_euler_blob_amr.jl                                             |           1      1     30.7s
      elixir_euler_sedov_blast_wave.jl                                     |   13            13     46.4s
      elixir_euler_sedov_blast_wave.jl (HLLE)                              |   13            13   1m44.9s
ERROR: LoadError: Some tests did not pass: 222 passed, 0 failed, 1 errored, 0 broken.
in expression starting at /Users/erik/git/Trixi.jl/test/test_tree_3d_part1.jl:1
Test Summary:     | Pass  Total     Time
TreeMesh3D Part 2 |  127    127  4m15.7s
Test Summary:                                              | Pass  Fail  Total     Time
TreeMesh3D Part 2                                          |  320     2    322  9m47.2s
  Examples 3D                                              |  308     2    310  9m46.9s
    MHD                                                    |  147          147  6m38.2s
    Lattice-Boltzmann                                      |  114          114  1m20.7s
    Linear scalar advection                                |    8     2     10    28.1s
      elixir_advection_extended.jl                         |    5            5    16.2s
      elixir_advection_extended.jl with periodic operators |    3     2      5    11.7s
    Compressible Euler                                     |   39           39  1m19.3s
  Additional tests in 3D                                   |   11           11     0.1s
ERROR: LoadError: Some tests did not pass: 320 passed, 2 failed, 0 errored, 0 broken.
in expression starting at /Users/erik/git/Trixi.jl/test/test_tree_3d_part3.jl:1
sloede commented 6 months ago

Thanks a lot for investigating this. It seems all be related to using some form of positivity limiting

efaulhaber commented 6 months ago

Most, if not all closure errors on Apple Silicon are caused by similar constructs as in #1462 in different indicators used for AMR and limiting.

efaulhaber commented 6 months ago

The current problems in 1D and 2D are caused by:

Note that I mentioned the 1D and 2D indicators separately because these are different codes that need to be refactored like I did in #1462.