Working with raw ICs with different file names in CIME Workflow

[ligiabernardet]

In the MRW App v1.1 documentation, we tell users that the way to use raw ICs with file names that differ from the default is to edit user_nl_ufsatm, for example, grib2_file_input_grid = gfs_3_20190829_0000_000.grb2 However, this is not working. Instead, the only way to use files with different names is by linking them to the name expected by the app: ln -s gfs_4_20190829_0000_000.grb gfs_3_20190829_0000_000.grb Does @uturuncoglu want to fix this behavior for the v1.1 release? Or do we keep it as is for the v1.1 release?

[uturuncoglu]

@ligiabernardet I am not sure about the correct implementation in the CIME interface at this point but I'll look at it. @jedwards4b is looking for changing data server list to match for the app update. In the mean time, if you could provide me a reproducer or just steps to reproduce the issue, that would be great

[ligiabernardet]

@uturuncoglu I have the case setup on Hera but I think you do not have access to it. So I will provide you with the steps.

• First you do a regular plain vanilla run with the App, for example using the default date and raw ICs (GRIB2 0.5 degree), which will access a file called gfs_4_YYYYMMDD_00HH_000.grb2.

• Then you obtain GRIB2 1.0 degree from the data server (which is called gfs_3_YYYYMMDD_00HH_000.grb2).

• Now you want to instruct the App to use this 1.0 degree file. My understanding is that we want to edit user_nl_ufsatm and add the line: grib2_file_input_grid = gfs_3_20190829_0000_000.grb2. However, this is not working. The current behavior is that this information is ignored and the App looks for file gfs_4_YYYYMMDD_00HH_000.grb2. Therefore, we have used a workaround by linking one file to another.

It just occurred to me that this could be a problem with chgres_cube and not with the App. But please see if you think the App is behaving appropriately in passing this variable to chgres_cube. Thanks.

[uturuncoglu]

@ligiabernardet could you also send me the link that I could use to download gfs_3 data.

[ligiabernardet]

My understanding is that it is https://www.ncei.noaa.gov/thredds/model/gfs.html

Also see https://github.com/ufs-community/ufs-mrweather-app/issues/161

On Thu, Jul 16, 2020 at 2:03 PM Ufuk Turunçoğlu notifications@github.com wrote:

@ligiabernardet https://github.com/ligiabernardet could you also send me the link that I could use to download gfs_3 data.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/ufs-community/ufs-mrweather-app/issues/159#issuecomment-659640834, or unsubscribe https://github.com/notifications/unsubscribe-auth/AE7WQARTGCDK73IELOTKMEDR35MH5ANCNFSM4OX6FZAQ .

[arunchawla-NOAA]

@GeorgeGayno-NOAA

How is the name of the input grib2 file provided to chgres_cube?

[GeorgeGayno-NOAA]

@GeorgeGayno-NOAA

How is the name of the input grib2 file provided to chgres_cube?

Through a namelist entry.

[uturuncoglu]

@GeorgeGayno-NOAA how do we have distinction between 0.5 deg. gfs-004 files and 1.0 deg gfs-003 files in chgres_cube?

[GeorgeGayno-NOAA]

@GeorgeGayno-NOAA how do we have distinction between 0.5 deg. gfs-004 files and 1.0 deg gfs-003 files in chgres_cube?

chgres_cube will automatically determine whether it is the 0.5-degree or 1.0-degree data.

[uturuncoglu]

@GeorgeGayno-NOAA Thanks for the information. @ligiabernardet Under CIME we were using input_type to have distinction between NEMSIO and GRIB2 but there is no way to tell CIME to use 1.0 or 0.5 deg. data. For example, if the input directory have both of them and if user wants to use a 1.0 deg. data it would be hard to use it. We might need to introduce new namelist variable just to be used internally by CIME. What do you think @jedwards4b @rsdunlapiv ?

[rsdunlapiv]

@uturuncoglu does this affect both the automatic download and the input to chgres? This seems like something the user would definitely want to be able to control. Alternatively, CIME could always default to 1.0deg and we could document how the user would manually stage the 0.5deg data and update the chgres namelist.

If you were to add a new CIME variable, what would it be called?

[jedwards4b]

@uturuncoglu I think we set a default and let the user know how to change it in user_nl_ufsatm. We don't need a new variable.

[uturuncoglu]

@rsdunlapiv Then we could only support for example automatic download for 1.0 deg. data. Is it fine for you @ligiabernardet? Currently there is no way to change it via user_nl_ufsatm unless we introduce new variable because the prefix of the file will change based on its resolution. This could be an option stored in env_run.xml or others. Do you have any suggestion @jedwards4b?

[ligiabernardet]

IMHO, auto-download for the 0.5 deg (gfs_4) files, as is currently done, is sufficient. I don't we need to auto-download 1 deg files. My understanding is that chgres_cube will detect the resolution of the GRIB2 file and process accordingly. The way a user can control which file is used is by the name of the file. In chgres_cube, the variable grib2_file_input_grid describes the name of the file to be used. When not used, the App should default to the gfs_4 files. If the user wants a different type of file, this variable can be customized. Just make sure variable grib2_file_input_grid is working through the CIME/App (it was not working for v1.0.0) so Ufuk and I resorted to linking files.

On Mon, Aug 17, 2020 at 3:01 PM Ufuk Turunçoğlu notifications@github.com wrote:

@rsdunlapiv https://github.com/rsdunlapiv Then we could only support for example automatic download for 1.0 deg. data. Is it fine for you @ligiabernardet https://github.com/ligiabernardet? Currently there is no way to change it via user_nl_ufsatm unless we introduce new variable because the prefix of the file will change based on its resolution. This could be an option stored in env_run.xml or others. Do you have any suggestion @jedwards4b https://github.com/jedwards4b?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/ufs-community/ufs-mrweather-app/issues/159#issuecomment-675112892, or unsubscribe https://github.com/notifications/unsubscribe-auth/AE7WQAWTTJIP7CHIQJTGI5LSBGLB5ANCNFSM4OX6FZAQ .

[uturuncoglu]

@ligiabernardet okay. let me work on the implementation that supports automatic download of both 0.5 and 1.0 deg. files. I could make 0.5 deg. as default and user might modify resolution_for_grib2 variable through the use of user_nl_ufsatm to use 1.0 deg. files and download them. What do you think?

[ligiabernardet]

@Ufuk Turuncoglu turuncu@ucar.edu It is fine with me if you want to proceed this way. For the v1.0.0 we did not have auto-downloads of 1.0 deg data, and that was not a problem. So, I don't think it is a requirement for this release. But of course it is nice to have. Question: what about 0.25 deg GRIB2 files? The App UG introduction mentions support for 0.25 degrees data. I have never used that. Should that be removed from the text?

On Mon, Aug 17, 2020 at 3:09 PM Ufuk Turunçoğlu notifications@github.com wrote:

@ligiabernardet https://github.com/ligiabernardet okay. let me work on the implementation that supports automatic download of both 0.5 and 1.0 deg. files. I could make 0.5 deg. as default and user might modify resolution_for_grib2 variable through the use of user_nl_ufsatm to use 1.0 deg. files and download them. What do you think?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/ufs-community/ufs-mrweather-app/issues/159#issuecomment-675116611, or unsubscribe https://github.com/notifications/unsubscribe-auth/AE7WQAXQAMLK5N5QHKPXJ4DSBGL6ZANCNFSM4OX6FZAQ .

[uturuncoglu]

@ligiabernardet it is done now. CIME could handle both 0.5 and 1.0 deg. files and it could also retrieve them from the server. I never try to use 0.25 deg. data. If we know the remote location and if chgres supports it we could also support it under CIME. Just by adding resolution_for_grib2 = "1.0" or resolution_for_grib2 = "0.5" you could have different file prefix gfs4 or gfs3. The resolution_for_grib2 is currently implemented as char and we need some modification to make it real with valid_values tag. This can be done later once I have also process NetCDF.

[ligiabernardet]

Great, thanks.

[ligiabernardet]

This has been fixed with the introduction of variable resolution_for_grib2, which can be customized by editing user_nl_ufsatm. Using this variable, the App can use GRIB2 files in 0.5 and 1.0 format without having to specify variable grib2_file_input_grid.