search

ufs-community / ufs-mrweather-app

UFS Medium-Range Weather Application
Other
23 stars 23 forks source link

FTP site needs to be updated with sample GFSv16 NetCDF data #156

Closed ligiabernardet closed 4 years ago

ligiabernardet commented 4 years ago

This website (https://ftp.emc.ncep.noaa.gov/EIB/UFS/) needs to be updated with sample GFSv16 NetCDF data https://ftp.emc.ncep.noaa.gov/EIB/UFS/ so we can develop and test the v1.1 App, as well as point the users to this sample dataset if they want to experiment with this new capability. @KateFriedman-NOAA I think you prepared the ftp site for the v1.0 release, would you also be doing this for the v1.1 release?

KateFriedman-NOAA commented 4 years ago

@ligiabernardet I would assume so, although this is the first mention of v1.1 to me so I haven't been tasked yet. @arunchawla-NOAA

Happy to assist though!

ligiabernardet commented 4 years ago

@arunchawla-NOAA We need to update ftp site (https://ftp.emc.ncep.noaa.gov/EIB/UFS/) with sample GFSv16 NetCDF data so we can develop and test the v1.1 App, as well as point the users to this sample dataset if they want to experiment with this new capability. Do you know if @KateFriedman-NOAA will helping in this capacity? Or is there someone else at EMC that has access to the NetCDF files?

arunchawla-NOAA commented 4 years ago

@KateFriedman-NOAA can you work with Fanglin and confirm with @GeorgeGayno-NOAA and get a sample set of NETCDF files that can be used. I think George has already tested that this works with chgres_cube (though we will need to confirm that it works with the release branch version). Will the 127 levels create a problem ?

GeorgeGayno-NOAA commented 4 years ago

@KateFriedman-NOAA can you work with Fanglin and confirm with @GeorgeGayno-NOAA and get a sample set of NETCDF files that can be used. I think George has already tested that this works with chgres_cube (though we will need to confirm that it works with the release branch version). Will the 127 levels create a problem ?

I have a sample dataset on Hera: /scratch1/NCEPDEV/da/George.Gayno/noscrub/chgres.gfsv16/gfs.20200202/00. I don't recall which parallel I got this data from. You may want data from a more recent parallel. Yes, it has 127 levels. But chgres_cube will handle 127 levels.

KateFriedman-NOAA commented 4 years ago

@arunchawla-NOAA @yangfanglin Sure thing...do folks want a newer data set than the February 2nd 2020 George has provided? If not, I will copy what George has on Hera to the ftp server.

@ligiabernardet Where would you like the data to land? A new folder in the "inputdata" folder? Or I can make a new folder under UFS for test data. Or do we want to start a new folder for release v1.1 stuff? All the files under the UFS folder are currently for v1.0. Lemme know your preference, thanks!

ligiabernardet commented 4 years ago

@Kate Friedman - NOAA Federal kate.howard@noaa.gov In my opinion, the 02/02/2020 date provided is fine.

A new folder under https://ftp.emc.ncep.noaa.gov/EIB/UFS/inputdata/ makes sense to me.

I looked through https://ftp.emc.ncep.noaa.gov/EIB/UFS and I think there is no need to make any changes except for a) adding the netCDF input data and b) updating file https://ftp.emc.ncep.noaa.gov/EIB/UFS/release_README to mention this new dataset.

Therefore, the updated https://ftp.emc.ncep.noaa.gov/EIB/UFS can serve for either v1.0 and v1.1.

On Mon, Aug 10, 2020 at 12:12 PM Kate Friedman notifications@github.com wrote:

@arunchawla-NOAA https://github.com/arunchawla-NOAA @yangfanglin https://github.com/yangfanglin Sure thing...do folks want a newer data set than the February 2nd 2020 George has provided? If not, I will copy what George has on Hera to the ftp server.

@ligiabernardet https://github.com/ligiabernardet Where would you like the data to land? A new folder in the "inputdata" folder https://ftp.emc.ncep.noaa.gov/EIB/UFS/inputdata/? Or I can make a new folder under UFS for test data. Or do we want to start a new folder for release v1.1 stuff? All the files under the UFS folder are currently for v1.0. Lemme know your preference, thanks!

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/ufs-community/ufs-mrweather-app/issues/156#issuecomment-671507746, or unsubscribe https://github.com/notifications/unsubscribe-auth/AE7WQARJNWDWH3EZ6RC4RR3SAA2ADANCNFSM4OJNBHQQ .

KateFriedman-NOAA commented 4 years ago

@ligiabernardet I've copied George's test data to the inputdata folder:

https://ftp.emc.ncep.noaa.gov/EIB/UFS/inputdata/202002/gfs.20200202/00/

@ligiabernardet @GeorgeGayno-NOAA Please provide the additional blurb (or updated README) for a new netcdf description in the release_README:

Thanks!

KateFriedman-NOAA commented 4 years ago

@ligiabernardet George provided updated text for the README via email. Have updated release_README to include the netcdf blurb. Thanks @GeorgeGayno-NOAA !

rsdunlapiv commented 4 years ago

@uturuncoglu Where do the ICs come from when you run the CIME regression test suite for the MR Weather app? Is it from the sample data provided here: ftp.emc.ncep.noaa.gov/EIB/UFS/ or the now defunct FTP site?  ftp://nomads.ncdc.noaa.gov

I am trying to see whether we need to fix the FTP site issue before anyone can run the regression test suite or if having the files on the sample data server (the first one above) is sufficient.

For reference, input data servers registered for CIME are here: https://github.com/ESMCI/cime/blob/master/config/ufs/config_inputdata.xml  

ligiabernardet commented 4 years ago

Thank you @KateFriedman-NOAA and @GeorgeGayno-NOAA for the update to the ftp site (https://ftp.emc.ncep.noaa.gov/EIB/UFS). It looks good to me.

uturuncoglu commented 4 years ago

@rsdunlapiv it depends on the order of the servers defined under CIME and their availability. The current order,

  1. ftp.emc.ncep.noaa.gov/EIB/UFS/
  2. https://nomads.ncdc.noaa.gov
  3. ftp://nomads.ncdc.noaa.gov

So, the test data is coming from ftp.emc.ncep.noaa.gov/EIB/UFS/ because it is the first server that file available.

ligiabernardet commented 4 years ago

All the necessary data is now in the FTP site.