search

NOAA-EMC / global-workflow

Global Superstructure/Workflow supporting the Global Forecast System (GFS)
https://global-workflow.readthedocs.io/en/latest
GNU Lesser General Public License v3.0
74 stars 165 forks source link

Product generation for GEFSv13 reforecast #1878

Closed EricSinsky-NOAA closed 1 month ago

EricSinsky-NOAA commented 11 months ago

What new functionality do you need?

There are a subset of variables that need to be saved on WCOSS2 for the GEFSv13 reforecast. These variables are needed by GEFSv13 reforecast major stakeholders. Functionalities that are needed include variable extraction from netcdf and grib2 files, daily-averaging, and data compression.

What are the requirements for the new functionality?

The wgrib2 utility is needed for data extraction from grib2 files, and to perform daily-averaging for a few GEFS upper atmosphere variables. The ncks utility is needed for data extraction from netcdf files. The bzip2 tool is needed to perform the data compression. For GEFS atmos, pgrb data in 0.25, 0.5 and 1.0 degree are needed as inputs. For GEFS ocean and ice, either the native resolution data or remapped 1 degree data are needed as inputs. GEFS wave and aerosol inputs will be needed as well.

Acceptance Criteria

This issue is considered complete when all stakeholder-requested variables for the GEFSv13 reforecast can be extracted, daily-averaged and compressed in a reasonable amount of time, and saved on WCOSS2.

Suggest a solution (optional)

In an effort to address this issue, a preliminary ex-script has been created to extract the stakeholder-requested variables from the atmosphere, ocean and ice model output. This preliminary script is similar to the script that was used for the the GEFSv12 reforecast. In this preliminary ex-script, extraction is performed using the wgrib2 utility for grib2 files and ncks for netcdf files. Furthermore, the daily-averaging in this script is performed using the wgrib2 utility, and compression is performed using bzip2.

DavidHuber-NOAA commented 1 month ago

@EricSinsky-NOAA Was this resolved by https://github.com/NOAA-EMC/global-workflow/pull/2680 or is there still more work to be done?

EricSinsky-NOAA commented 1 month ago

This task has not been tested successfully yet when REPLAY_ICS=YES. There are PRs that will allow this task to run successfully when REPLAY_ICS=YES (PR #2778, PR #2755). There are issues already associated with those PRs, so I think this issue can be closed as complete.

DavidHuber-NOAA commented 1 month ago

Sounds good, thanks @EricSinsky-NOAA. Closing.