search

NOAA-EMC / JEDI-T2O

JEDI Transition to Operations activities.
GNU Lesser General Public License v2.1
5 stars 4 forks source link

Add utility script to overwrite RESTART fields from FV3-JEDI analysis #9

Closed CoryMartin-NOAA closed 2 years ago

CoryMartin-NOAA commented 2 years ago

Closes #8

This adds a utility python script to overwrite FV3 RESTART fields using arrays from a FV3-JEDI analysis file in the FV3 RESTART format.

CoryMartin-NOAA commented 2 years ago

@RussTreadon-NOAA thanks for timing tests. Wow, those are slow. These are for the Gaussian files and not the RESTART tile files? For our workflow we will be using the RESTART files and can use cfp to parallelize, but I think you've already shown that we are going to need a different approach when this becomes operational.

RussTreadon-NOAA commented 2 years ago

Correct, the timings are for gaussian grid files. Here are timings to execute update_restart.py for tile1 from $COMROOTp3/gfs/prod/gdas.20211221/06/atmos/RESTART/ for the indicated filetypes:

These timings are significantly faster. This makes sense since tiles are being processed.

On Tue, Dec 21, 2021 at 12:38 PM Cory Martin @.***> wrote:

@RussTreadon-NOAA https://github.com/RussTreadon-NOAA thanks for timing tests. Wow, those are slow. These are for the Gaussian files and not the RESTART tile files? For our workflow we will be using the RESTART files and can use cfp to parallelize, but I think you've already shown that we are going to need a different approach when this becomes operational.

— Reply to this email directly, view it on GitHub https://github.com/NOAA-EMC/JEDI-T2O/pull/9#issuecomment-998970536, or unsubscribe https://github.com/notifications/unsubscribe-auth/AGNN63ZZXFI6J33QVNGI74DUSC3SNANCNFSM5KOXAA3A . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

You are receiving this because you were mentioned.Message ID: @.***>

CoryMartin-NOAA commented 2 years ago

Thanks @RussTreadon-NOAA , I bet a big part of that runtime for the Gaussian grid files is because of the netCDF compression.

RussTreadon-NOAA commented 2 years ago

I agree. Compression costs cpu cycles.

On Tue, Dec 21, 2021 at 2:23 PM Cory Martin @.***> wrote:

Thanks @RussTreadon-NOAA https://github.com/RussTreadon-NOAA , I bet a big part of that runtime for the Gaussian grid files is because of the netCDF compression.

— Reply to this email directly, view it on GitHub https://github.com/NOAA-EMC/JEDI-T2O/pull/9#issuecomment-999034261, or unsubscribe https://github.com/notifications/unsubscribe-auth/AGNN63ZKJH5DANF5ONUIR43USDH2NANCNFSM5KOXAA3A . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

You are receiving this because you were mentioned.Message ID: @.***>