search

geoschem / gcpy

Python toolkit for GEOS-Chem. Contains basic plotting scripts, plus the suite of GEOS-Chem benchmarking utilities.
https://gcpy.readthedocs.io
Other
51 stars 24 forks source link

Now flag GEOS-Chem Classic/GCHP timers that differ by more than 10% in the benchmark timing tables #332

Closed yantosca closed 3 months ago

yantosca commented 3 months ago

Name and Institution (Required)

Name: Bob Yantosca Institution: Harvard + GCST

Describe the update

We have updated the benchmark_scrape_gcclassic_timers.py and benchmark_scrape_gchp_timers.py scripts to flag individual GEOS-Chem Classic and GCHP timers that vary by more than +/- 10%. This will help us to identify performance bottlenecks much more easily.

Expected changes

Benchmark timing tables now will contain an asterisk after timers that vary by more than +/- 10%.

GEOS-Chem Classic:

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%% GEOS-Chem Classic Benchmark Timing Information
%%%
%%% Ref = gcc-14.4.2
%%% Dev = gcc-14.4.1
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

Timer                              Ref [s]             Dev [s]         % Diff
-------------------------------------------------------------------------------
GEOS-Chem                        28458.415           21173.062     -2.560e+01 *
HEMCO                             2307.501            2287.382     -8.719e-01
All chemistry                    15052.359            8087.397     -4.627e+01 *
=> Gas-phase chem                 5536.890            5295.887     -4.353e+00
=> Photolysis                     7353.315             691.712     -9.059e+01 *
=> Aerosol chem                   1597.813            1598.909      6.859e-02
=> Linearized chem                  30.455              29.045     -4.630e+00
Transport                         1689.448            1660.802     -1.696e+00
Convection                        2490.201            2413.583     -3.077e+00
Boundary layer mixing             2403.692            2271.712     -5.491e+00
Dry deposition                      51.891              51.624     -5.145e-01
Wet deposition                     774.040             755.015     -2.458e+00
Diagnostics                       2367.490            2446.812      3.350e+00
Unit conversions                  1422.235            1267.408     -1.089e+01 *

^^^ note: Ref and Dev were swapped in this output inadvertently, but you get the idea.

GCHP

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%% GCHP Benchmark Timing Information
%%%
%%% Ref = gchp-14.4.2
%%% Dev = gchp-14.4.1
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

GCHPchem Timer                     Ref [s]             Dev [s]         % Diff
-------------------------------------------------------------------------------
GCHPchem                          7017.550            7032.760      2.167e-01
--SetService                         0.280               0.270     -3.571e+00
----generic                          0.000               0.000            nan
--Initialize                        30.660              30.090     -1.859e+00
----INITIALIZE                      30.660              30.090     -1.859e+00
------generic                       12.930              13.140      1.624e+00
--Record                             0.020               0.020      0.000e+00
----generic                          0.010               0.010      0.000e+00
--Run                             6947.260            6981.810      4.973e-01
----GenRunMine                    6947.220            6981.760      4.972e-01
------RUN                         6907.250            6936.510      4.236e-01
--------CP_BFRE                      3.070               3.360      9.446e+00
--------DO_CHEM                   6869.460            6897.840      4.131e-01
----------GC_CONV                  523.710             501.900     -4.165e+00
----------GC_DRYDEP                  8.800               8.770     -3.409e-01
----------GC_EMIS                  184.410             186.890      1.345e+00
----------GC_TURB                  619.830             595.020     -4.003e+00
----------GC_CHEM                 4736.540            4751.700      3.201e-01
----------GC_WETDEP                445.050             468.660      5.305e+00
----------GC_DIAGN                  36.980              37.190      5.679e-01
--------CP_AFTR                      0.010               0.010      0.000e+00
--Finalize                           3.220               3.120     -3.106e+00
----generic                          3.210               3.120     -2.804e+00

Summary                            Ref [s]             Dev [s]         % Diff
-------------------------------------------------------------------------------
All                              25606.259           25584.746     -8.401e-02
--SetService                         1.522               1.430     -6.045e+00
----GCHP                             1.499               1.407     -6.137e+00
------GCHPctmEnv                     0.001               0.001      0.000e+00
------GCHPchem                       0.456               0.450     -1.316e+00
------DYNAMICS                       0.981               0.896     -8.665e+00
----HIST                             0.000               0.000            nan
----EXTDATA                          0.000               0.000            nan
--Initialize                        98.315             104.915      6.713e+00
----GCHP                            69.100              75.827      9.735e+00
------GCHPctmEnv                     0.013               0.013      0.000e+00
------GCHPchem                      61.798              65.188      5.486e+00
------DYNAMICS                       5.680               9.107      6.033e+01 *
----HIST                             7.027               7.170      2.035e+00
----EXTDATA                         21.023              21.338      1.498e+00
--Run                            25502.983           25475.018     -1.097e-01
----EXTDATA                       1067.898            1074.784      6.448e-01
----GCHP                         17065.536           17103.029      2.197e-01
------GCHPctmEnv                    44.852              45.314      1.030e+00
------GCHPchem                   11228.451           11255.168      2.379e-01
------DYNAMICS                    5790.771            5801.109      1.785e-01
----HIST                           142.520             145.059      1.782e+00
--Finalize                           3.292               3.189     -3.129e+00
----GCHP                             3.272               3.175     -2.965e+00
------GCHPctmEnv                     0.005               0.006      2.000e+01 *
------GCHPchem                       3.261               3.163     -3.005e+00
------DYNAMICS                       0.003               0.003      0.000e+00
----HIST                             0.007               0.006     -1.429e+01 *
----EXTDATA                          0.010               0.005     -5.000e+01 *

Reference(s)

If this is a science update, please provide a literature citation.

Related Github Issue

lizziel commented 3 months ago

This updates looks good but I'm curious why we use scientific notation for the % diff. It would be easier to scan, and might not even need the *s, if we list the percentages plus maybe just one decimal place.

lizziel commented 3 months ago

Having a column for difference would also be good to have.

yantosca commented 3 months ago

Thanks @lizziel. I can implement a floating point format rather than the scientific notation for % diff.

yantosca commented 3 months ago

@lizziel: I reverted to floating point format in commit b07327d. See updated output below:

GCClassic

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%% GEOS-Chem Classic Benchmark Timing Information
%%%
%%% Ref = gcc-14.4.1
%%% Dev = gcc-14.4.2
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

Timer                              Ref [s]             Dev [s]         % Diff
-------------------------------------------------------------------------------
GEOS-Chem                        21173.062           28458.415         34.409 *
HEMCO                             2287.382            2307.501          0.880
All chemistry                     8087.397           15052.359         86.121 *
=> Gas-phase chem                 5295.887            5536.890          4.551
=> Photolysis                      691.712            7353.315        963.060 *
=> Aerosol chem                   1598.909            1597.813         -0.069
=> Linearized chem                  29.045              30.455          4.855
Transport                         1660.802            1689.448          1.725
Convection                        2413.583            2490.201          3.174
Boundary layer mixing             2271.712            2403.692          5.810
Dry deposition                      51.624              51.891          0.517
Wet deposition                     755.015             774.040          2.520
Diagnostics                       2446.812            2367.490         -3.242
Unit conversions                  1267.408            1422.235         12.216 *

GCHP

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%% GCHP Benchmark Timing Information
%%%
%%% Ref = gchp-14.4.1
%%% Dev = gchp-14.4.2
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

GCHPchem Timer                     Ref [s]             Dev [s]         % Diff
-------------------------------------------------------------------------------
GCHPchem                          7032.760            7017.550         -0.216
--SetService                         0.270               0.280          3.704
----generic                          0.000               0.000            nan
--Initialize                        30.090              30.660          1.894
----INITIALIZE                      30.090              30.660          1.894
------generic                       13.140              12.930         -1.598
--Record                             0.020               0.020          0.000
----generic                          0.010               0.010          0.000
--Run                             6981.810            6947.260         -0.495
----GenRunMine                    6981.760            6947.220         -0.495
------RUN                         6936.510            6907.250         -0.422
--------CP_BFRE                      3.360               3.070         -8.631
--------DO_CHEM                   6897.840            6869.460         -0.411
----------GC_CONV                  501.900             523.710          4.345
----------GC_DRYDEP                  8.770               8.800          0.342
----------GC_EMIS                  186.890             184.410         -1.327
----------GC_TURB                  595.020             619.830          4.170
----------GC_CHEM                 4751.700            4736.540         -0.319
----------GC_WETDEP                468.660             445.050         -5.038
----------GC_DIAGN                  37.190              36.980         -0.565
--------CP_AFTR                      0.010               0.010          0.000
--Finalize                           3.120               3.220          3.205
----generic                          3.120               3.210          2.885

Summary                            Ref [s]             Dev [s]         % Diff
-------------------------------------------------------------------------------
All                              25584.746           25606.259          0.084
--SetService                         1.430               1.522          6.434
----GCHP                             1.407               1.499          6.539
------GCHPctmEnv                     0.001               0.001          0.000
------GCHPchem                       0.450               0.456          1.333
------DYNAMICS                       0.896               0.981          9.487
----HIST                             0.000               0.000            nan
----EXTDATA                          0.000               0.000            nan
--Initialize                       104.915              98.315         -6.291
----GCHP                            75.827              69.100         -8.872
------GCHPctmEnv                     0.013               0.013          0.000
------GCHPchem                      65.188              61.798         -5.200
------DYNAMICS                       9.107               5.680        -37.630 *
----HIST                             7.170               7.027         -1.994
----EXTDATA                         21.338              21.023         -1.476
--Run                            25475.018           25502.983          0.110
----EXTDATA                       1074.784            1067.898         -0.641
----GCHP                         17103.029           17065.536         -0.219
------GCHPctmEnv                    45.314              44.852         -1.020
------GCHPchem                   11255.168           11228.451         -0.237
------DYNAMICS                    5801.109            5790.771         -0.178
----HIST                           145.059             142.520         -1.750
--Finalize                           3.189               3.292          3.230
----GCHP                             3.175               3.272          3.055
------GCHPctmEnv                     0.006               0.005        -16.667 *
------GCHPchem                       3.163               3.261          3.098
------DYNAMICS                       0.003               0.003          0.000
----HIST                             0.006               0.007         16.667 *
----EXTDATA                          0.005               0.010        100.000 *