Adding another APCP variable (accumulation since model initialized)

LinlinCui-NOAA commented 4 months ago

Description

This PR added a utility function to subset grib2 files for North America area.
Added another variable APCP to represent an accumulation of the predicted precipitation amounts since model initialized.

Linked Issues

Closes https://github.com/NOAA-EMC/graphcast/issues/32

Blocking Dependencies

None

Anticipated Changes

None

Input data

[x] No changes are expected to input data.
[ ] Changes are expected to input data:
- [ ] New input data.
- [ ] Updated input data.

Needed libraries

None

LinlinCui-NOAA commented 4 months ago

@SadeghTabas-NOAA Please let me know the preferred subsetting bounding box. Thanks.

SadeghTabas-NOAA commented 4 months ago

@SadeghTabas-NOAA Please let me know the preferred subsetting bounding box. Thanks. @LinlinCui-NOAA Please consider an approximate bound like this figure, if we needed we can correct the values. Thanks

SadeghTabas-NOAA commented 4 months ago

@LinlinCui-NOAA Please do your modifications in utility scritp, do not update run_graphcast.py. These are experimental and we don't want to update the main code. Thanks

SadeghTabas-NOAA commented 4 months ago

@LinlinCui-NOAA Please do your modifications in utility scritp, do not update run_graphcast.py. These are experimental and we don't want to update the main code. Thanks

The upload function uploads every file in the output directory to s3, please have a look and see if it uploads the grib2 files in the subset folder in the right directory. I'll send you the credentials for S3 bucket. Thanks

LinlinCui-NOAA commented 4 months ago

The upload function uploads every file in the output directory to s3, please have a look and see if it uploads the grib2 files in the subset folder in the right directory. I'll send you the credentials for S3 bucket. Thanks

@SadeghTabas-NOAA I am not sure if subfolder "north_america" must be pre-created to upload files there. But the script does upload files in the subfolder:

root is forecasts_13_levels
uploading forecasts_13_levels/graphcastgfs.t00z.pgrb2.0p25.f102 to graphcastgfs.20240215/00/forecasts_13_levels/graphcastgfs.t00z.pgrb2.0p25.f102
uploading forecasts_13_levels/graphcastgfs.t00z.pgrb2.0p25.f186 to graphcastgfs.20240215/00/forecasts_13_levels/graphcastgfs.t00z.pgrb2.0p25.f186
.
.
.
root is forecasts_13_levels/north_america
uploading forecasts_13_levels/north_america/graphcastgfs.t00z.pgrb2.0p25.f102 to graphcastgfs.20240215/00/forecasts_13_levels/north_america/graphcastgfs.t00z.pgrb2.0p25.f102
uploading forecasts_13_levels/north_america/graphcastgfs.t00z.pgrb2.0p25.f186 to graphcastgfs.20240215/00/forecasts_13_levels/north_america/graphcastgfs.t00z.pgrb2.0p25.f186

SadeghTabas-NOAA commented 4 months ago

The upload function uploads every file in the output directory to s3, please have a look and see if it uploads the grib2 files in the subset folder in the right directory. I'll send you the credentials for S3 bucket. Thanks

@SadeghTabas-NOAA I am not sure if subfolder "north_america" must be pre-created to upload files there. But the script does upload files in the subfolder:
root is forecasts_13_levels
uploading forecasts_13_levels/graphcastgfs.t00z.pgrb2.0p25.f102 to graphcastgfs.20240215/00/forecasts_13_levels/graphcastgfs.t00z.pgrb2.0p25.f102
uploading forecasts_13_levels/graphcastgfs.t00z.pgrb2.0p25.f186 to graphcastgfs.20240215/00/forecasts_13_levels/graphcastgfs.t00z.pgrb2.0p25.f186
.
.
.
root is forecasts_13_levels/north_america
uploading forecasts_13_levels/north_america/graphcastgfs.t00z.pgrb2.0p25.f102 to graphcastgfs.20240215/00/forecasts_13_levels/north_america/graphcastgfs.t00z.pgrb2.0p25.f102
uploading forecasts_13_levels/north_america/graphcastgfs.t00z.pgrb2.0p25.f186 to graphcastgfs.20240215/00/forecasts_13_levels/north_america/graphcastgfs.t00z.pgrb2.0p25.f186

@LinlinCui-NOAA As we talked about today, we will not upload the subset results to S3 and we won't modify the code as well, please just make a sample subset of the grib2 file for f006 (over North America) and share it with @junwang-noaa. So, your current PR should only reflect the new APCP message in the grib2 files. Please update the PR title as well. Thanks

SadeghTabas-NOAA commented 4 months ago

@LinlinCui-NOAA Also, if you overwrite something on S3, please let me know so I can regenerate those. Thanks

LinlinCui-NOAA commented 4 months ago

The upload function uploads every file in the output directory to s3, please have a look and see if it uploads the grib2 files in the subset folder in the right directory. I'll send you the credentials for S3 bucket. Thanks

@SadeghTabas-NOAA I am not sure if subfolder "north_america" must be pre-created to upload files there. But the script does upload files in the subfolder:
root is forecasts_13_levels
uploading forecasts_13_levels/graphcastgfs.t00z.pgrb2.0p25.f102 to graphcastgfs.20240215/00/forecasts_13_levels/graphcastgfs.t00z.pgrb2.0p25.f102
uploading forecasts_13_levels/graphcastgfs.t00z.pgrb2.0p25.f186 to graphcastgfs.20240215/00/forecasts_13_levels/graphcastgfs.t00z.pgrb2.0p25.f186
.
.
.
root is forecasts_13_levels/north_america
uploading forecasts_13_levels/north_america/graphcastgfs.t00z.pgrb2.0p25.f102 to graphcastgfs.20240215/00/forecasts_13_levels/north_america/graphcastgfs.t00z.pgrb2.0p25.f102
uploading forecasts_13_levels/north_america/graphcastgfs.t00z.pgrb2.0p25.f186 to graphcastgfs.20240215/00/forecasts_13_levels/north_america/graphcastgfs.t00z.pgrb2.0p25.f186
@LinlinCui-NOAA As we talked about today, we will not upload the subset results to S3 and we won't modify the code as well, please just make a sample subset of the grib2 file for f006 (over North America) and share it with @junwang-noaa. So, your current PR should only reflect the new APCP message in the grib2 files. Please update the PR title as well. Thanks

@SadeghTabas-NOAA This was posted before the meeting. The latest commit has commented out subsetting function.

LinlinCui-NOAA commented 4 months ago

@LinlinCui-NOAA Also, if you overwrite something on S3, please let me know so I can regenerate those. Thanks

@SadeghTabas-NOAA I did not upload files to S3, just printed file names for both sides to check.

NOAA-EMC / graphcast