search

NOAA-EMC / GSI

Gridpoint Statistical Interpolation
GNU Lesser General Public License v3.0
66 stars 149 forks source link

Updates to build and run on Orion Rocky 9 #764

Closed RussTreadon-NOAA closed 3 months ago

RussTreadon-NOAA commented 3 months ago

Description This PR updates NOAA-EMC/GSI to build and run on Orion Rocky 9.

Resolves #754

Type of change

How Has This Been Tested? Install on Orion and run ctests with results (all tests Passed) posted in issue #754.

Checklist

RussTreadon-NOAA commented 3 months ago

Open PR in draft mode given increased wall times observed for gsi.x and enkf.x when run on Orion Rocky 9.

RussTreadon-NOAA commented 3 months ago

Slowness of gsi.x and enkf.x on Orion Rocky 9 remains unexplained but will change this PR to Ready for review to invite feedback on proposed changes.

RussTreadon-NOAA commented 3 months ago

Agreed, @DavidHuber-NOAA ! The gsi.x and enkf.x slowdown on Orion Rocky 9 does not make sense to me.

aerorahul commented 3 months ago

@RussTreadon-NOAA Can this PR be merged with the knowledge that GSI is running in degraded status on Orion? This will allow the global-workflow to proceed for Orion+Rocky8. When the source of the degradation is identified on Orion, we can update the submodule pointer for Orion+Rocky8

Tagging @CatherineThomas-NOAA for awareness and discussions on the GFSv17 project tag-up.

RussTreadon-NOAA commented 3 months ago

@aerorahul , let me check with the GSI Review team

@ShunLiu-NOAA , @CoryMartin-NOAA , and @hu5970 : Are we OK merging this PR into GSI develop even though ctests show gsi.x and enkf.x run approximately 2x slower on Orion Rocky 9 and Orion Centos 7? @aerorahul explains above why this question is being asked.

I'm reluctant to merge since do so may lessen the urgency of addressing the 2x slowdown. That said, we don't want NOAA-EMC/GSI to become the roadblock for completion of the g-w transition to Orion Rocky 9 (see issue #2694)

CoryMartin-NOAA commented 3 months ago

@RussTreadon-NOAA I share your concerns but I don't think we have a choice. It's either "runs slow" or "not at all", so I think we have to go with the former.

aerorahul commented 3 months ago

@aerorahul , let me check with the GSI Review team

@ShunLiu-NOAA , @CoryMartin-NOAA , and @hu5970 : Are we OK merging this PR into GSI develop even though ctests show gsi.x and enkf.x run approximately 2x slower on Orion Rocky 9 and Orion Centos 7? @aerorahul explains above why this question is being asked.

I'm reluctant to merge since do so may lessen the urgency of addressing the 2x slowdown. That said, we don't want NOAA-EMC/GSI to become the roadblock for completion of the g-w transition to Orion Rocky 9 (see issue #2694)

Just open another issue to report the performance degradation on Orion after the upgrade and follow the development there.

RussTreadon-NOAA commented 3 months ago

Actions already taken reporting gsi.x and enkf.x slowdown on Orion Rocky-9:

Someone will need to follow up on the ticket and issue. doing so will likely require trying various things until the problem is resolved.