search

EarthWorksOrg / EarthWorks

Other
3 stars 2 forks source link

Update Derecho config to use NVHPC/24.3 #41

Closed gdicker1 closed 6 months ago

gdicker1 commented 6 months ago

Incorporate changes to EWOrg/ccs_config_cesm to use the ncarenv/23.09 software stack and nvhpc/24.3 on the Derecho supercomputer.

gdicker1 commented 6 months ago

NOTE: this PR will be moved out of draft once the related PRs are tested, merged, tagged, and this PR branch is updated to use a tag instead.

(i.e. if all the PRs mentioned in this comment below are merged and boxes are checked, this PR should be ready for final review and/or merge)

gdicker1 commented 6 months ago

Sub / Required PRs in Externals:

Other work:

gdicker1 commented 6 months ago

Since I have been able to merge these changes and successfully run the compsets FHS94, FKESSLER, F2000climo, QPC6, FullyCoupled on CPUs, and FullyCoupled on GPUs on mpasa120 grids, this PR will soon be moved out of draft and merged onto the develop branch.

dazlich commented 6 months ago

Hi Dylan,

My travels are over and I am back into the swing of things. I checked out the develop branch and tried to run F2000climo with MPASSI (2000_CAM60_CLM50%SP_MPASSI%PRES_DOCN%DOM_MOSART_CISM2%NOEVOLVE_SWAV). It failed to compile with both the intel and nvhpc compilers (didn’t try gnu). It was framework conflicts at the load step that killed it, specifically subroutines mpas_rbf_interp_func_3D_plane_sca_lin_dir_neu_comp_coeffs and mpas_rbf_interp_func_3D_plane_vec_const_tan_neu_comp_coeffs in operators/mpas_rbf_interpolation.F. This puzzled me since they are in modules that are differently named which I thought should have resolved the problem.

Anyway, these routines are called nowhere in the EW code. I commented them out in the mpas-framework version of the file. Then everything compiled and ran fine. Intel versions restarted and of course the atmosphere restart failed with nvhpc. I also tested the fully coupled model. These were one day tests at 120km/32layers, writing full atm history and all restarts.

run directories: F2000climo (2000_CAM60_CLM50%SP_MPASSI%PRES_DOCN%DOM_MOSART_CISM2%NOEVOLVE_SWAV): /glade/derecho/scratch/dazlich/EWv2_F2000climo_MPASSI.120.L32.derecho.intel.256 /glade/derecho/scratch/dazlich/EWv2_F2000climo_MPASSI.120.L32.derecho.nvhpc.256

B2000: /glade/derecho/scratch/dazlich/EWv2.1_B2000.120.L32.derecho.intel.256 /glade/derecho/scratch/dazlich/EWv2.1_B2000.120.L32.nvhpc.256

To summarize, this is a one file fix for this problem.

As a side note, it would be nice to define tags for the above compsets, say F2000climoEW and B2000EW

On Apr 19, 2024, at 4:07 PM, G. Dylan Dickerson @.***> wrote:

Caution: EXTERNAL Sender

Merged #41https://github.com/EarthWorksOrg/EarthWorks/pull/41 into develop.

— Reply to this email directly, view it on GitHubhttps://github.com/EarthWorksOrg/EarthWorks/pull/41#event-12545422958, or unsubscribehttps://github.com/notifications/unsubscribe-auth/ADS3XWAM7PPK3WBMCZVHMWDY6GISRAVCNFSM6AAAAABGJ2AWP6VHI2DSMVQWIX3LMV45UABCJFZXG5LFIV3GK3TUJZXXI2LGNFRWC5DJN5XDWMJSGU2DKNBSGI4TKOA. You are receiving this because your review was requested.Message ID: @.***>

gdicker1 commented 6 months ago

@dazlich, thanks for pointing that out! Do you think you could push the commenting out of this code in a new PR? (Also if you have the error message from one of those builds that would be appreciated!)

And agreed on the tags/short-names of B2000EW and F2000climoEW, they would be nice to add along with FullyCoupledEW (or something like it).

dazlich commented 6 months ago

Can do!

On Apr 23, 2024, at 4:32 PM, G. Dylan Dickerson @.***> wrote:

Caution: EXTERNAL Sender

@dazlichhttps://github.com/dazlich, thanks for pointing that out! Do you think you could push the commenting out of this code in a new PR? (Also if you have the error message from one of those builds that would be appreciated!)

And agreed on the tags/short-names of B2000EW and F2000climoEW, they would be nice to add along with FullyCoupledEW (or something like it).

— Reply to this email directly, view it on GitHubhttps://github.com/EarthWorksOrg/EarthWorks/pull/41#issuecomment-2073584763, or unsubscribehttps://github.com/notifications/unsubscribe-auth/ADS3XWBDEHWE2XCIQZ2UKIDY63ORBAVCNFSM6AAAAABGJ2AWP6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDANZTGU4DINZWGM. You are receiving this because you were mentioned.Message ID: @.***>