Remove unnecessary Kokkos::single from compute_ppm

mfdeakin-sandia commented 6 years ago

This appears to work on White now

mfdeakin-sandia commented 6 years ago

Performance comparison with a single rank on a node of white, with ne=4, qsize=35, ndays=2

HommeTime_stats.master.1
prim_main_loop                             1        1 1.000000e+00   4.949342e+00     4.949 (     0      0)     4.949 (     0      0)
tl-ae U3-5stage_timestep                   1        1 2.880000e+02   7.224100e-01     0.722 (     0      0)     0.722 (     0      0)
tl-ae advance_hypervis_dp                  1        1 2.880000e+02   6.836495e-01     0.684 (     0      0)     0.684 (     0      0)
tl-s prim_advec_tracers_remap              1        1 2.880000e+02   2.901469e+00     2.901 (     0      0)     2.901 (     0      0)
tl-at prim_advec_tracers_remap_RK2         1        1 2.880000e+02   2.901349e+00     2.901 (     0      0)     2.901 (     0      0)
tl-sc vertical_remap                       1        1 9.600000e+01   5.705938e-01     0.571 (     0      0)     0.571 (     0      0)
Remap Thickness Functor                    1        1 9.600000e+01   9.619474e-03     0.010 (     0      0)     0.010 (     0      0)
Remap Scale States Functor                 1        1 9.600000e+01   4.398346e-03     0.004 (     0      0)     0.004 (     0      0)
Remap Compute Grids Functor                1        1 9.600000e+01   1.410937e-02     0.014 (     0      0)     0.014 (     0      0)
Remap Compute Remap Functor                1        1 9.600000e+01   5.330272e-01     0.533 (     0      0)     0.533 (     0      0)
Remap Rescale States Functor               1        1 9.600000e+01   4.520655e-03     0.005 (     0      0)     0.005 (     0      0)

HommeTime_stats.master.2
prim_main_loop                             1        1 1.000000e+00   4.957392e+00     4.957 (     0      0)     4.957 (     0      0)
tl-ae U3-5stage_timestep                   1        1 2.880000e+02   7.213140e-01     0.721 (     0      0)     0.721 (     0      0)
tl-ae advance_hypervis_dp                  1        1 2.880000e+02   6.830001e-01     0.683 (     0      0)     0.683 (     0      0)
tl-s prim_advec_tracers_remap              1        1 2.880000e+02   2.897520e+00     2.898 (     0      0)     2.898 (     0      0)
tl-at prim_advec_tracers_remap_RK2         1        1 2.880000e+02   2.897403e+00     2.897 (     0      0)     2.897 (     0      0)
tl-sc vertical_remap                       1        1 9.600000e+01   5.708969e-01     0.571 (     0      0)     0.571 (     0      0)
Remap Thickness Functor                    1        1 9.600000e+01   9.878635e-03     0.010 (     0      0)     0.010 (     0      0)
Remap Scale States Functor                 1        1 9.600000e+01   4.478455e-03     0.004 (     0      0)     0.004 (     0      0)
Remap Compute Grids Functor                1        1 9.600000e+01   1.411796e-02     0.014 (     0      0)     0.014 (     0      0)
Remap Compute Remap Functor                1        1 9.600000e+01   5.328875e-01     0.533 (     0      0)     0.533 (     0      0)
Remap Rescale States Functor               1        1 9.600000e+01   4.540205e-03     0.005 (     0      0)     0.005 (     0      0)

HommeTime_stats.master.3
prim_main_loop                             1        1 1.000000e+00   4.945650e+00     4.946 (     0      0)     4.946 (     0      0)
tl-ae U3-5stage_timestep                   1        1 2.880000e+02   7.183356e-01     0.718 (     0      0)     0.718 (     0      0)
tl-ae advance_hypervis_dp                  1        1 2.880000e+02   6.815047e-01     0.682 (     0      0)     0.682 (     0      0)
tl-s prim_advec_tracers_remap              1        1 2.880000e+02   2.892132e+00     2.892 (     0      0)     2.892 (     0      0)
tl-at prim_advec_tracers_remap_RK2         1        1 2.880000e+02   2.892018e+00     2.892 (     0      0)     2.892 (     0      0)
tl-sc vertical_remap                       1        1 9.600000e+01   5.696325e-01     0.570 (     0      0)     0.570 (     0      0)
Remap Thickness Functor                    1        1 9.600000e+01   9.657621e-03     0.010 (     0      0)     0.010 (     0      0)
Remap Scale States Functor                 1        1 9.600000e+01   4.410028e-03     0.004 (     0      0)     0.004 (     0      0)
Remap Compute Grids Functor                1        1 9.600000e+01   1.410007e-02     0.014 (     0      0)     0.014 (     0      0)
Remap Compute Remap Functor                1        1 9.600000e+01   5.319192e-01     0.532 (     0      0)     0.532 (     0      0)
Remap Rescale States Functor               1        1 9.600000e+01   4.602909e-03     0.005 (     0      0)     0.005 (     0      0)

HommeTime_stats.master.4
prim_main_loop                             1        1 1.000000e+00   4.956190e+00     4.956 (     0      0)     4.956 (     0      0)
tl-ae U3-5stage_timestep                   1        1 2.880000e+02   7.198551e-01     0.720 (     0      0)     0.720 (     0      0)
tl-ae advance_hypervis_dp                  1        1 2.880000e+02   6.862381e-01     0.686 (     0      0)     0.686 (     0      0)
tl-s prim_advec_tracers_remap              1        1 2.880000e+02   2.896113e+00     2.896 (     0      0)     2.896 (     0      0)
tl-at prim_advec_tracers_remap_RK2         1        1 2.880000e+02   2.896000e+00     2.896 (     0      0)     2.896 (     0      0)
tl-sc vertical_remap                       1        1 9.600000e+01   5.697227e-01     0.570 (     0      0)     0.570 (     0      0)
Remap Thickness Functor                    1        1 9.600000e+01   9.668589e-03     0.010 (     0      0)     0.010 (     0      0)
Remap Scale States Functor                 1        1 9.600000e+01   4.442692e-03     0.004 (     0      0)     0.004 (     0      0)
Remap Compute Grids Functor                1        1 9.600000e+01   1.413298e-02     0.014 (     0      0)     0.014 (     0      0)
Remap Compute Remap Functor                1        1 9.600000e+01   5.320165e-01     0.532 (     0      0)     0.532 (     0      0)
Remap Rescale States Functor               1        1 9.600000e+01   4.548311e-03     0.005 (     0      0)     0.005 (     0      0)

HommeTime_stats.rm_ppm_single.1
prim_main_loop                             1        1 1.000000e+00   4.737081e+00     4.737 (     0      0)     4.737 (     0      0)
tl-ae U3-5stage_timestep                   1        1 2.880000e+02   7.194693e-01     0.719 (     0      0)     0.719 (     0      0)
tl-ae advance_hypervis_dp                  1        1 2.880000e+02   6.832917e-01     0.683 (     0      0)     0.683 (     0      0)
tl-s prim_advec_tracers_remap              1        1 2.880000e+02   2.940022e+00     2.940 (     0      0)     2.940 (     0      0)
tl-at prim_advec_tracers_remap_RK2         1        1 2.880000e+02   2.939922e+00     2.940 (     0      0)     2.940 (     0      0)
tl-sc vertical_remap                       1        1 9.600000e+01   3.095670e-01     0.310 (     0      0)     0.310 (     0      0)
Remap Thickness Functor                    1        1 9.600000e+01   9.611845e-03     0.010 (     0      0)     0.010 (     0      0)
Remap Scale States Functor                 1        1 9.600000e+01   4.430056e-03     0.004 (     0      0)     0.004 (     0      0)
Remap Compute Grids Functor                1        1 9.600000e+01   1.406622e-02     0.014 (     0      0)     0.014 (     0      0)
Remap Compute Remap Functor                1        1 9.600000e+01   2.718105e-01     0.272 (     0      0)     0.272 (     0      0)
Remap Rescale States Functor               1        1 9.600000e+01   4.611969e-03     0.005 (     0      0)     0.005 (     0      0)

HommeTime_stats.rm_ppm_single.2
prim_main_loop                             1        1 1.000000e+00   4.686712e+00     4.687 (     0      0)     4.687 (     0      0)
tl-ae U3-5stage_timestep                   1        1 2.880000e+02   7.189929e-01     0.719 (     0      0)     0.719 (     0      0)
tl-ae advance_hypervis_dp                  1        1 2.880000e+02   6.813204e-01     0.681 (     0      0)     0.681 (     0      0)
tl-s prim_advec_tracers_remap              1        1 2.880000e+02   2.893331e+00     2.893 (     0      0)     2.893 (     0      0)
tl-at prim_advec_tracers_remap_RK2         1        1 2.880000e+02   2.893208e+00     2.893 (     0      0)     2.893 (     0      0)
tl-sc vertical_remap                       1        1 9.600000e+01   3.088970e-01     0.309 (     0      0)     0.309 (     0      0)
Remap Thickness Functor                    1        1 9.600000e+01   9.595871e-03     0.010 (     0      0)     0.010 (     0      0)
Remap Scale States Functor                 1        1 9.600000e+01   4.397154e-03     0.004 (     0      0)     0.004 (     0      0)
Remap Compute Grids Functor                1        1 9.600000e+01   1.407027e-02     0.014 (     0      0)     0.014 (     0      0)
Remap Compute Remap Functor                1        1 9.600000e+01   2.713315e-01     0.271 (     0      0)     0.271 (     0      0)
Remap Rescale States Functor               1        1 9.600000e+01   4.594088e-03     0.005 (     0      0)     0.005 (     0      0)

HommeTime_stats.rm_ppm_single.3
prim_main_loop                             1        1 1.000000e+00   4.706897e+00     4.707 (     0      0)     4.707 (     0      0)
tl-ae U3-5stage_timestep                   1        1 2.880000e+02   7.231956e-01     0.723 (     0      0)     0.723 (     0      0)
tl-ae advance_hypervis_dp                  1        1 2.880000e+02   6.896915e-01     0.690 (     0      0)     0.690 (     0      0)
tl-s prim_advec_tracers_remap              1        1 2.880000e+02   2.899728e+00     2.900 (     0      0)     2.900 (     0      0)
tl-at prim_advec_tracers_remap_RK2         1        1 2.880000e+02   2.899582e+00     2.900 (     0      0)     2.900 (     0      0)
tl-sc vertical_remap                       1        1 9.600000e+01   3.095093e-01     0.310 (     0      0)     0.310 (     0      0)
Remap Thickness Functor                    1        1 9.600000e+01   9.728432e-03     0.010 (     0      0)     0.010 (     0      0)
Remap Scale States Functor                 1        1 9.600000e+01   4.549026e-03     0.005 (     0      0)     0.005 (     0      0)
Remap Compute Grids Functor                1        1 9.600000e+01   1.419091e-02     0.014 (     0      0)     0.014 (     0      0)
Remap Compute Remap Functor                1        1 9.600000e+01   2.713857e-01     0.271 (     0      0)     0.271 (     0      0)
Remap Rescale States Functor               1        1 9.600000e+01   4.646063e-03     0.005 (     0      0)     0.005 (     0      0)

HommeTime_stats.rm_ppm_single.4
prim_main_loop                             1        1 1.000000e+00   4.689117e+00     4.689 (     0      0)     4.689 (     0      0)
tl-ae U3-5stage_timestep                   1        1 2.880000e+02   7.191291e-01     0.719 (     0      0)     0.719 (     0      0)
tl-ae advance_hypervis_dp                  1        1 2.880000e+02   6.840129e-01     0.684 (     0      0)     0.684 (     0      0)
tl-s prim_advec_tracers_remap              1        1 2.880000e+02   2.892758e+00     2.893 (     0      0)     2.893 (     0      0)
tl-at prim_advec_tracers_remap_RK2         1        1 2.880000e+02   2.892622e+00     2.893 (     0      0)     2.893 (     0      0)
tl-sc vertical_remap                       1        1 9.600000e+01   3.088999e-01     0.309 (     0      0)     0.309 (     0      0)
Remap Thickness Functor                    1        1 9.600000e+01   9.649992e-03     0.010 (     0      0)     0.010 (     0      0)
Remap Scale States Functor                 1        1 9.600000e+01   4.474640e-03     0.004 (     0      0)     0.004 (     0      0)
Remap Compute Grids Functor                1        1 9.600000e+01   1.408386e-02     0.014 (     0      0)     0.014 (     0      0)
Remap Compute Remap Functor                1        1 9.600000e+01   2.711520e-01     0.271 (     0      0)     0.271 (     0      0)
Remap Rescale States Functor               1        1 9.600000e+01   4.555941e-03     0.005 (     0      0)     0.005 (     0      0)

master and rm_ppm_single runs were interleaved

mfdeakin-sandia commented 6 years ago

This passes nightly tests on ride and bowman, and dev tests on shepard. I will test on skybridge tomorrow. This appears to have a slight (unexpected, so verifying) performance benefit (3% overall) on bowman with ne=8, qsize=35, ndays=4:

HommeTime_stats.master.1
prim_main_loop                            64       64 6.400000e+01   2.195759e+03    34.310 (     8      0)    34.307 (    28      0)
tl-ae U3-5stage_timestep                  64       64 3.686400e+04   2.621676e+02     4.862 (    30      0)     3.424 (     8      0)
tl-ae advance_hypervis_dp                 64       64 3.686400e+04   2.177693e+02     3.444 (    58      0)     3.384 (    48      0)
tl-s prim_advec_tracers_remap             64       64 3.686400e+04   1.469102e+03    23.638 (     8      0)    22.145 (    30      0)
tl-at prim_advec_tracers_remap_RK2        64       64 3.686400e+04   1.468886e+03    23.635 (     8      0)    22.141 (    30      0)
tl-sc vertical_remap                      64       64 1.228800e+04   2.171702e+02     3.463 (    44      0)     3.359 (    56      0)
Remap Thickness Functor                   64       64 1.228800e+04   1.489665e+00     0.024 (     6      0)     0.022 (    61      0)
Remap Scale States Functor                64       64 1.228800e+04   7.867690e-01     0.013 (    39      0)     0.011 (     2      0)
Remap Compute Grids Functor               64       64 1.228800e+04   8.527969e+00     0.137 (    52      0)     0.129 (     8      0)
Remap Compute Remap Functor               64       64 1.228800e+04   2.041988e+02     3.258 (    44      0)     3.160 (    56      0)
Remap Rescale States Functor              64       64 1.228800e+04   1.122507e+00     0.019 (     7      0)     0.017 (    29      0)

HommeTime_stats.master.2
prim_main_loop                            64       64 6.400000e+01   2.201904e+03    34.407 (    58      0)    34.403 (    28      0)
tl-ae U3-5stage_timestep                  64       64 3.686400e+04   2.656623e+02     4.923 (    30      0)     3.424 (     8      0)
tl-ae advance_hypervis_dp                 64       64 3.686400e+04   2.187265e+02     3.460 (    62      0)     3.395 (    35      0)
tl-s prim_advec_tracers_remap             64       64 3.686400e+04   1.470934e+03    23.711 (     8      0)    22.155 (    30      0)
tl-at prim_advec_tracers_remap_RK2        64       64 3.686400e+04   1.470712e+03    23.707 (     8      0)    22.151 (    30      0)
tl-sc vertical_remap                      64       64 1.228800e+04   2.170143e+02     3.460 (    51      0)     3.355 (    38      0)
Remap Thickness Functor                   64       64 1.228800e+04   1.491414e+00     0.025 (    54      0)     0.022 (     8      0)
Remap Scale States Functor                64       64 1.228800e+04   7.885790e-01     0.014 (     6      0)     0.011 (    10      0)
Remap Compute Grids Functor               64       64 1.228800e+04   8.514999e+00     0.138 (    52      0)     0.129 (     8      0)
Remap Compute Remap Functor               64       64 1.228800e+04   2.040784e+02     3.254 (    51      0)     3.156 (    38      0)
Remap Rescale States Functor              64       64 1.228800e+04   1.125978e+00     0.019 (    13      0)     0.017 (    30      0)

HommeTime_stats.master.3
prim_main_loop                            64       64 6.400000e+01   2.201295e+03    34.397 (    58      0)    34.393 (    27      0)
tl-ae U3-5stage_timestep                  64       64 3.686400e+04   2.664574e+02     4.975 (    30      0)     3.407 (     8      0)
tl-ae advance_hypervis_dp                 64       64 3.686400e+04   2.186488e+02     3.506 (    58      0)     3.383 (    48      0)
tl-s prim_advec_tracers_remap             64       64 3.686400e+04   1.469571e+03    23.748 (     8      0)    22.089 (    30      0)
tl-at prim_advec_tracers_remap_RK2        64       64 3.686400e+04   1.469350e+03    23.745 (     8      0)    22.086 (    30      0)
tl-sc vertical_remap                      64       64 1.228800e+04   2.170606e+02     3.456 (    44      0)     3.356 (    14      0)
Remap Thickness Functor                   64       64 1.228800e+04   1.488806e+00     0.024 (     4      0)     0.022 (     2      0)
Remap Scale States Functor                64       64 1.228800e+04   7.852150e-01     0.014 (    43      0)     0.011 (     8      0)
Remap Compute Grids Functor               64       64 1.228800e+04   8.519231e+00     0.138 (    52      0)     0.128 (     8      0)
Remap Compute Remap Functor               64       64 1.228800e+04   2.041068e+02     3.254 (    44      0)     3.155 (    14      0)
Remap Rescale States Functor              64       64 1.228800e+04   1.122532e+00     0.019 (     7      0)     0.017 (    30      0)

HommeTime_stats.master.4
prim_main_loop                            64       64 6.400000e+01   2.196942e+03    34.329 (    36      0)    34.325 (    27      0)
tl-ae U3-5stage_timestep                  64       64 3.686400e+04   2.633150e+02     4.844 (    30      0)     3.454 (     8      0)
tl-ae advance_hypervis_dp                 64       64 3.686400e+04   2.181744e+02     3.453 (    58      0)     3.384 (    15      0)
tl-s prim_advec_tracers_remap             64       64 3.686400e+04   1.468719e+03    23.616 (     8      0)    22.168 (    30      0)
tl-at prim_advec_tracers_remap_RK2        64       64 3.686400e+04   1.468503e+03    23.613 (     8      0)    22.165 (    30      0)
tl-sc vertical_remap                      64       64 1.228800e+04   2.171793e+02     3.466 (    51      0)     3.358 (    61      0)
Remap Thickness Functor                   64       64 1.228800e+04   1.487122e+00     0.024 (     4      0)     0.022 (     8      0)
Remap Scale States Functor                64       64 1.228800e+04   7.837300e-01     0.014 (    39      0)     0.011 (     5      0)
Remap Compute Grids Functor               64       64 1.228800e+04   8.524191e+00     0.138 (    52      0)     0.130 (     8      0)
Remap Compute Remap Functor               64       64 1.228800e+04   2.042216e+02     3.261 (    51      0)     3.162 (    61      0)
Remap Rescale States Functor              64       64 1.228800e+04   1.124913e+00     0.019 (     8      0)     0.017 (    52      0)

HommeTime_stats.rm_ppm_single.1
prim_main_loop                            64       64 6.400000e+01   2.120706e+03    33.138 (    33      0)    33.133 (    27      0)
tl-ae U3-5stage_timestep                  64       64 3.686400e+04   2.632431e+02     4.926 (    30      0)     3.436 (     8      0)
tl-ae advance_hypervis_dp                 64       64 3.686400e+04   2.174456e+02     3.449 (    58      0)     3.376 (    27      0)
tl-s prim_advec_tracers_remap             64       64 3.686400e+04   1.466160e+03    23.590 (     8      0)    22.092 (    30      0)
tl-at prim_advec_tracers_remap_RK2        64       64 3.686400e+04   1.465944e+03    23.586 (     8      0)    22.088 (    30      0)
tl-sc vertical_remap                      64       64 1.228800e+04   1.443126e+02     2.285 (    43      0)     2.240 (    61      0)
Remap Thickness Functor                   64       64 1.228800e+04   1.506256e+00     0.025 (     4      0)     0.022 (     2      0)
Remap Scale States Functor                64       64 1.228800e+04   7.825590e-01     0.014 (    43      0)     0.011 (    58      0)
Remap Compute Grids Functor               64       64 1.228800e+04   8.520673e+00     0.137 (    52      0)     0.129 (    55      0)
Remap Compute Remap Functor               64       64 1.228800e+04   1.313548e+02     2.077 (    43      0)     2.042 (    16      0)
Remap Rescale States Functor              64       64 1.228800e+04   1.124931e+00     0.019 (     7      0)     0.017 (    51      0)

HommeTime_stats.rm_ppm_single.2
prim_main_loop                            64       64 6.400000e+01   2.123196e+03    33.177 (    36      0)    33.172 (    25      0)
tl-ae U3-5stage_timestep                  64       64 3.686400e+04   2.642343e+02     4.921 (    30      0)     3.452 (     8      0)
tl-ae advance_hypervis_dp                 64       64 3.686400e+04   2.180476e+02     3.449 (    58      0)     3.387 (    15      0)
tl-s prim_advec_tracers_remap             64       64 3.686400e+04   1.466882e+03    23.605 (     8      0)    22.111 (    30      0)
tl-at prim_advec_tracers_remap_RK2        64       64 3.686400e+04   1.466664e+03    23.602 (     8      0)    22.108 (    30      0)
tl-sc vertical_remap                      64       64 1.228800e+04   1.444895e+02     2.280 (    52      0)     2.240 (    38      0)
Remap Thickness Functor                   64       64 1.228800e+04   1.507111e+00     0.025 (    54      0)     0.022 (    61      0)
Remap Scale States Functor                64       64 1.228800e+04   7.838370e-01     0.014 (    52      0)     0.011 (     5      0)
Remap Compute Grids Functor               64       64 1.228800e+04   8.533307e+00     0.138 (    52      0)     0.130 (     2      0)
Remap Compute Remap Functor               64       64 1.228800e+04   1.315150e+02     2.073 (    48      0)     2.039 (    38      0)
Remap Rescale States Functor              64       64 1.228800e+04   1.123631e+00     0.019 (    36      0)     0.017 (    46      0)

HommeTime_stats.rm_ppm_single.3
prim_main_loop                            64       64 6.400000e+01   2.120112e+03    33.129 (    19      0)    33.124 (    27      0)
tl-ae U3-5stage_timestep                  64       64 3.686400e+04   2.637084e+02     4.932 (    30      0)     3.442 (     8      0)
tl-ae advance_hypervis_dp                 64       64 3.686400e+04   2.174151e+02     3.428 (    58      0)     3.374 (    28      0)
tl-s prim_advec_tracers_remap             64       64 3.686400e+04   1.465150e+03    23.580 (     8      0)    22.075 (    30      0)
tl-at prim_advec_tracers_remap_RK2        64       64 3.686400e+04   1.464936e+03    23.576 (     8      0)    22.072 (    30      0)
tl-sc vertical_remap                      64       64 1.228800e+04   1.442805e+02     2.276 (    52      0)     2.239 (    20      0)
Remap Thickness Functor                   64       64 1.228800e+04   1.508901e+00     0.025 (     4      0)     0.022 (     8      0)
Remap Scale States Functor                64       64 1.228800e+04   7.863550e-01     0.014 (    43      0)     0.011 (    55      0)
Remap Compute Grids Functor               64       64 1.228800e+04   8.531173e+00     0.137 (    52      0)     0.130 (    58      0)
Remap Compute Remap Functor               64       64 1.228800e+04   1.312947e+02     2.069 (    52      0)     2.037 (    20      0)
Remap Rescale States Functor              64       64 1.228800e+04   1.123969e+00     0.019 (     7      0)     0.016 (    48      0)

HommeTime_stats.rm_ppm_single.4
prim_main_loop                            64       64 6.400000e+01   2.126971e+03    33.236 (    36      0)    33.232 (    29      0)
tl-ae U3-5stage_timestep                  64       64 3.686400e+04   2.676657e+02     5.039 (    30      0)     3.430 (     8      0)
tl-ae advance_hypervis_dp                 64       64 3.686400e+04   2.173446e+02     3.477 (    58      0)     3.365 (    15      0)
tl-s prim_advec_tracers_remap             64       64 3.686400e+04   1.468184e+03    23.719 (     8      0)    22.075 (    30      0)
tl-at prim_advec_tracers_remap_RK2        64       64 3.686400e+04   1.467969e+03    23.716 (     8      0)    22.071 (    30      0)
tl-sc vertical_remap                      64       64 1.228800e+04   1.442429e+02     2.274 (    52      0)     2.238 (    14      0)
Remap Thickness Functor                   64       64 1.228800e+04   1.507726e+00     0.025 (     4      0)     0.022 (     8      0)
Remap Scale States Functor                64       64 1.228800e+04   7.840750e-01     0.014 (    39      0)     0.011 (     8      0)
Remap Compute Grids Functor               64       64 1.228800e+04   8.534283e+00     0.137 (    39      0)     0.130 (     8      0)
Remap Compute Remap Functor               64       64 1.228800e+04   1.312830e+02     2.067 (    43      0)     2.037 (    31      0)
Remap Rescale States Functor              64       64 1.228800e+04   1.121617e+00     0.019 (     1      0)     0.016 (    28      0)

mfdeakin-sandia commented 6 years ago

Shepard comparison appears neutral/potentially slightly worse with ne=4, qsize=35, ndays=4:

HommeTime_stats.master.1
prim_main_loop                            32       32 3.200000e+01   8.737607e+02    27.305 (     0      0)    27.305 (    11      0)
tl-ae U3-5stage_timestep                  32       32 1.843200e+04   6.871333e+01     4.293 (    21      0)     1.035 (     6      0)
tl-ae advance_hypervis_dp                 32       32 1.843200e+04   2.994486e+01     0.946 (    30      0)     0.925 (     7      0)
tl-s prim_advec_tracers_remap             32       32 1.843200e+04   7.291517e+02    23.862 (     6      0)    20.678 (    21      0)
tl-at prim_advec_tracers_remap_RK2        32       32 1.843200e+04   7.291117e+02    23.861 (     6      0)    20.677 (    21      0)
tl-sc vertical_remap                      32       32 6.144000e+03   3.562811e+01     1.168 (    15      0)     1.084 (    23      0)
Remap Thickness Functor                   32       32 6.144000e+03   4.397166e-01     0.021 (    15      0)     0.010 (    18      0)
Remap Scale States Functor                32       32 6.144000e+03   2.246063e-01     0.014 (    15      0)     0.003 (    23      0)
Remap Compute Grids Functor               32       32 6.144000e+03   2.102585e+00     0.102 (    15      0)     0.047 (    23      0)
Remap Compute Remap Functor               32       32 6.144000e+03   3.253628e+01     1.037 (    22      0)     1.008 (    10      0)
Remap Rescale States Functor              32       32 6.144000e+03   2.228884e-01     0.008 (     0      0)     0.007 (    25      0)

HommeTime_stats.master.2
prim_main_loop                            32       32 3.200000e+01   8.726374e+02    27.270 (     0      0)    27.270 (    25      0)
tl-ae U3-5stage_timestep                  32       32 1.843200e+04   6.849703e+01     4.310 (    21      0)     1.048 (     6      0)
tl-ae advance_hypervis_dp                 32       32 1.843200e+04   3.013889e+01     0.952 (    30      0)     0.932 (     5      0)
tl-s prim_advec_tracers_remap             32       32 1.843200e+04   7.278940e+02    23.806 (     6      0)    20.617 (    21      0)
tl-at prim_advec_tracers_remap_RK2        32       32 1.843200e+04   7.278509e+02    23.805 (     6      0)    20.616 (    21      0)
tl-sc vertical_remap                      32       32 6.144000e+03   3.563842e+01     1.169 (    14      0)     1.079 (    28      0)
Remap Thickness Functor                   32       32 6.144000e+03   4.452025e-01     0.021 (    11      0)     0.010 (    23      0)
Remap Scale States Functor                32       32 6.144000e+03   2.268802e-01     0.014 (    11      0)     0.003 (    23      0)
Remap Compute Grids Functor               32       32 6.144000e+03   2.108664e+00     0.099 (    11      0)     0.047 (    23      0)
Remap Compute Remap Functor               32       32 6.144000e+03   3.252897e+01     1.032 (    22      0)     1.005 (     9      0)
Remap Rescale States Functor              32       32 6.144000e+03   2.232977e-01     0.008 (     1      0)     0.007 (    27      0)

HommeTime_stats.master.3
prim_main_loop                            32       32 3.200000e+01   8.722681e+02    27.259 (     0      0)    27.258 (    11      0)
tl-ae U3-5stage_timestep                  32       32 1.843200e+04   6.955696e+01     4.316 (    21      0)     1.049 (     6      0)
tl-ae advance_hypervis_dp                 32       32 1.843200e+04   3.057956e+01     0.967 (    30      0)     0.946 (     5      0)
tl-s prim_advec_tracers_remap             32       32 1.843200e+04   7.261445e+02    23.791 (     6      0)    20.582 (    21      0)
tl-at prim_advec_tracers_remap_RK2        32       32 1.843200e+04   7.261050e+02    23.790 (     6      0)    20.581 (    21      0)
tl-sc vertical_remap                      32       32 6.144000e+03   3.562392e+01     1.176 (    12      0)     1.083 (    27      0)
Remap Thickness Functor                   32       32 6.144000e+03   4.389247e-01     0.020 (    15      0)     0.010 (    23      0)
Remap Scale States Functor                32       32 6.144000e+03   2.232599e-01     0.014 (    15      0)     0.003 (    18      0)
Remap Compute Grids Functor               32       32 6.144000e+03   2.091088e+00     0.101 (    11      0)     0.047 (    23      0)
Remap Compute Remap Functor               32       32 6.144000e+03   3.254489e+01     1.036 (    12      0)     1.006 (    27      0)
Remap Rescale States Functor              32       32 6.144000e+03   2.234919e-01     0.008 (     6      0)     0.007 (    10      0)

HommeTime_stats.master.4
prim_main_loop                            32       32 3.200000e+01   8.729287e+02    27.279 (     0      0)    27.279 (    25      0)
tl-ae U3-5stage_timestep                  32       32 1.843200e+04   6.836315e+01     4.295 (    21      0)     1.041 (     6      0)
tl-ae advance_hypervis_dp                 32       32 1.843200e+04   3.010457e+01     0.954 (    30      0)     0.930 (     8      0)
tl-s prim_advec_tracers_remap             32       32 1.843200e+04   7.283976e+02    23.822 (     6      0)    20.639 (    21      0)
tl-at prim_advec_tracers_remap_RK2        32       32 1.843200e+04   7.283582e+02    23.821 (     6      0)    20.638 (    21      0)
tl-sc vertical_remap                      32       32 6.144000e+03   3.559987e+01     1.173 (    11      0)     1.082 (    28      0)
Remap Thickness Functor                   32       32 6.144000e+03   4.412566e-01     0.021 (    11      0)     0.010 (    23      0)
Remap Scale States Functor                32       32 6.144000e+03   2.226421e-01     0.014 (    11      0)     0.003 (    23      0)
Remap Compute Grids Functor               32       32 6.144000e+03   2.085484e+00     0.101 (    11      0)     0.047 (    23      0)
Remap Compute Remap Functor               32       32 6.144000e+03   3.252386e+01     1.028 (    22      0)     1.006 (     4      0)
Remap Rescale States Functor              32       32 6.144000e+03   2.241230e-01     0.008 (     0      0)     0.007 (    27      0)

HommeTime_stats.rm_ppm_single.1
prim_main_loop                            32       32 3.200000e+01   8.755246e+02    27.360 (     0      0)    27.360 (    13      0)
tl-ae U3-5stage_timestep                  32       32 1.843200e+04   6.853247e+01     4.271 (    21      0)     1.033 (     6      0)
tl-ae advance_hypervis_dp                 32       32 1.843200e+04   3.008810e+01     0.950 (    30      0)     0.931 (     5      0)
tl-s prim_advec_tracers_remap             32       32 1.843200e+04   7.309029e+02    23.923 (     6      0)    20.745 (    21      0)
tl-at prim_advec_tracers_remap_RK2        32       32 1.843200e+04   7.308613e+02    23.922 (     6      0)    20.745 (    21      0)
tl-sc vertical_remap                      32       32 6.144000e+03   3.566047e+01     1.176 (    15      0)     1.075 (    18      0)
Remap Thickness Functor                   32       32 6.144000e+03   4.376241e-01     0.021 (    15      0)     0.010 (    18      0)
Remap Scale States Functor                32       32 6.144000e+03   2.238449e-01     0.014 (    15      0)     0.003 (    18      0)
Remap Compute Grids Functor               32       32 6.144000e+03   2.093430e+00     0.101 (    11      0)     0.047 (    18      0)
Remap Compute Remap Functor               32       32 6.144000e+03   3.258055e+01     1.036 (    22      0)     1.006 (    18      0)
Remap Rescale States Functor              32       32 6.144000e+03   2.230985e-01     0.008 (     0      0)     0.007 (    10      0)

HommeTime_stats.rm_ppm_single.2
prim_main_loop                            32       32 3.200000e+01   8.769835e+02    27.406 (     0      0)    27.406 (    25      0)
tl-ae U3-5stage_timestep                  32       32 1.843200e+04   6.871850e+01     4.311 (    21      0)     1.033 (     6      0)
tl-ae advance_hypervis_dp                 32       32 1.843200e+04   3.019336e+01     0.963 (    30      0)     0.929 (     8      0)
tl-s prim_advec_tracers_remap             32       32 1.843200e+04   7.319451e+02    23.958 (     6      0)    20.748 (    21      0)
tl-at prim_advec_tracers_remap_RK2        32       32 1.843200e+04   7.319057e+02    23.957 (     6      0)    20.747 (    21      0)
tl-sc vertical_remap                      32       32 6.144000e+03   3.568692e+01     1.161 (    15      0)     1.084 (    28      0)
Remap Thickness Functor                   32       32 6.144000e+03   4.410455e-01     0.019 (    15      0)     0.011 (    26      0)
Remap Scale States Functor                32       32 6.144000e+03   2.264747e-01     0.013 (    14      0)     0.003 (    23      0)
Remap Compute Grids Functor               32       32 6.144000e+03   2.097900e+00     0.098 (    14      0)     0.047 (    23      0)
Remap Compute Remap Functor               32       32 6.144000e+03   3.259723e+01     1.029 (    31      0)     1.011 (    28      0)
Remap Rescale States Functor              32       32 6.144000e+03   2.233663e-01     0.008 (     1      0)     0.007 (    27      0)

HommeTime_stats.rm_ppm_single.3
prim_main_loop                            32       32 3.200000e+01   8.739455e+02    27.311 (     0      0)    27.311 (     7      0)
tl-ae U3-5stage_timestep                  32       32 1.843200e+04   6.881764e+01     4.300 (    21      0)     1.022 (     6      0)
tl-ae advance_hypervis_dp                 32       32 1.843200e+04   3.011805e+01     0.955 (    30      0)     0.929 (     6      0)
tl-s prim_advec_tracers_remap             32       32 1.843200e+04   7.289426e+02    23.884 (     6      0)    20.663 (    21      0)
tl-at prim_advec_tracers_remap_RK2        32       32 1.843200e+04   7.289034e+02    23.883 (     6      0)    20.662 (    21      0)
tl-sc vertical_remap                      32       32 6.144000e+03   3.570675e+01     1.177 (    15      0)     1.084 (    28      0)
Remap Thickness Functor                   32       32 6.144000e+03   4.385276e-01     0.022 (    15      0)     0.010 (    18      0)
Remap Scale States Functor                32       32 6.144000e+03   2.238488e-01     0.015 (    15      0)     0.003 (    26      0)
Remap Compute Grids Functor               32       32 6.144000e+03   2.103088e+00     0.107 (    15      0)     0.046 (    26      0)
Remap Compute Remap Functor               32       32 6.144000e+03   3.261683e+01     1.034 (    21      0)     1.010 (    29      0)
Remap Rescale States Functor              32       32 6.144000e+03   2.238711e-01     0.008 (     6      0)     0.007 (    14      0)

HommeTime_stats.rm_ppm_single.4
prim_main_loop                            32       32 3.200000e+01   8.759420e+02    27.373 (     0      0)    27.373 (     2      0)
tl-ae U3-5stage_timestep                  32       32 1.843200e+04   6.888372e+01     4.297 (    21      0)     1.075 (     6      0)
tl-ae advance_hypervis_dp                 32       32 1.843200e+04   3.098512e+01     0.983 (    30      0)     0.957 (    12      0)
tl-s prim_advec_tracers_remap             32       32 1.843200e+04   7.300060e+02    23.861 (     6      0)    20.703 (    20      0)
tl-at prim_advec_tracers_remap_RK2        32       32 1.843200e+04   7.299649e+02    23.860 (     6      0)    20.702 (    20      0)
tl-sc vertical_remap                      32       32 6.144000e+03   3.570281e+01     1.178 (    15      0)     1.083 (    17      0)
Remap Thickness Functor                   32       32 6.144000e+03   4.407698e-01     0.020 (    15      0)     0.010 (    23      0)
Remap Scale States Functor                32       32 6.144000e+03   2.256609e-01     0.014 (    15      0)     0.003 (    23      0)
Remap Compute Grids Functor               32       32 6.144000e+03   2.110836e+00     0.100 (    15      0)     0.047 (    23      0)
Remap Compute Remap Functor               32       32 6.144000e+03   3.260146e+01     1.046 (    25      0)     1.006 (     6      0)
Remap Rescale States Functor              32       32 6.144000e+03   2.231150e-01     0.008 (     1      0)     0.007 (    27      0)

E3SM-Project / HOMMEXX

Remove unnecessary Kokkos::single from compute_ppm #277