We need a plan for connecting CIME to the public input data sources

[mvertens]

In order for CIME to be extended to support different forecast initial conditions flexibly, we need to understand the directory structure in the FTP site that will be made available. We need to understand the requirements for CIME to obtain this data and the list of files that will be needed. A major concern is the potentially large size of some of the files - and we need to determine if they need to be manually downloaded or have CIME download them automatically as part of its workflow.

[uturuncoglu]

Currently, i am getting them from the directory that i used for the prototype version of workflow.

[uturuncoglu]

CHGRES also required to

global_hyblev.l65.txt

So, maybe it could be good to create POST and PRE directories and place required fixed input files there. Any idea?

[arunchawla-NOAA]

@GeorgeGayno-NOAA and @LarissaReames-NOAA

Kate set up an ftp server with all the fix files needed for the model. Can you check to see if all the files needed for chgres are also there? If not can you let Kate know where to add them for, or alternatively place them in a directory there? Location of ftp server below

https://ftp.emc.ncep.noaa.gov/EIB/UFS/

[GeorgeGayno-NOAA]

@GeorgeGayno-NOAA and @LarissaReames-NOAA

Kate set up an ftp server with all the fix files needed for the model. Can you check to see if all the files needed for chgres are also there? If not can you let Kate know where to add them for, or alternatively place them in a directory there? Location of ftp server below

https://ftp.emc.ncep.noaa.gov/EIB/UFS/

The files needed for chgres are there.

[uturuncoglu]

I think we need to open this again. Following issues are not solved yet.

1 - We need to extract the tar file under https://ftp.emc.ncep.noaa.gov/EIB/UFS/simple-test-case/

2 - global_hyblev.l65.txt is in https://ftp.emc.ncep.noaa.gov/EIB/UFS/global/fix/fix_am.v20191213/ that is fine but we don't have following files and those are resolution dependent

C96.facsf..nc C96.maximum_snow_albedo..nc C96.slope_type..nc C96.snowfree_albedo..nc C96.soil_type..nc C96.substrate_temperature..nc C96.vegetation_greenness..nc C96.vegetation_type..nc

There are some files under this directory but those are in grib format. I think it would be better to put those netcdf files under

https://ftp.emc.ncep.noaa.gov/EIB/UFS/global/fix/fix_fv3_gmted2010.v20191213/

based on their resolution.

[GeorgeGayno-NOAA]

I think we need to open this again. Following issues are not solved yet.

1 - We need to extract the tar file under https://ftp.emc.ncep.noaa.gov/EIB/UFS/simple-test-case/

2 - global_hyblev.l65.txt is in https://ftp.emc.ncep.noaa.gov/EIB/UFS/global/fix/fix_am.v20191213/ that is fine but we don't have following files and those are resolution dependent

C96.facsf._.nc C96.maximum_snowalbedo..nc C96.slopetype..nc C96.snowfreealbedo..nc C96.soiltype..nc C96.substratetemperature..nc C96.vegetationgreenness..nc C96.vegetationtype..nc

The above files are under the ./fix_sfc subdirectory.

There are some files under this directory but those are in grib format. I think it would be better to put those netcdf files under

https://ftp.emc.ncep.noaa.gov/EIB/UFS/global/fix/fix_fv3_gmted2010.v20191213/

based on their resolution.

[uturuncoglu]

Okay. I could see them now. Thanks @GeorgeGayno-NOAA. We still need to extract the tar file.

[KateFriedman-NOAA]

@uturuncoglu I have unpacked the tarball:

https://ftp.emc.ncep.noaa.gov/EIB/UFS/simple-test-case/

I also moved the gzipped tarball up one level so it wasn't within the unpacked folder. Let me know if I should put it back down within the simple-test-case folder. The path to that tarball is now:

https://ftp.emc.ncep.noaa.gov/EIB/UFS/simple-test-case.tar.gz

[emc.glopara@vm-lnx-emcrzdm01 UFS]$ pwd
/home/ftp/emc/EIB/UFS
[emc.glopara@vm-lnx-emcrzdm01 UFS]$ ll
total 128464
drwxr-xr-x. 3 emc.glopara emc      4096 Dec 18 14:01 global
drwxr-xr-x. 4 emc.glopara emc      4096 Dec 30 15:08 simple-test-case
-rw-r--r--. 1 emc.glopara emc 131018461 Dec 20 19:15 simple-test-case.tar.gz
[emc.glopara@vm-lnx-emcrzdm01 UFS]$ ll simple-test-case
total 66816
-rw-r--r--. 1 emc.glopara emc  3135393 Dec 20 19:13 aerosol.dat
-rw-r--r--. 1 emc.glopara emc  3111408 Dec 20 19:13 CFSR.SEAICE.1982.2012.monthly.clim.grb
-rw-r--r--. 1 emc.glopara emc    24484 Dec 20 19:13 co2historicaldata_2016.txt
-rw-r--r--. 1 emc.glopara emc        0 Dec 20 19:13 data_table
-rw-r--r--. 1 emc.glopara emc    22919 Dec 20 19:13 diag_table
-rw-r--r--. 1 emc.glopara emc      678 Dec 20 19:13 field_table
-rw-r--r--. 1 emc.glopara emc  1394712 Dec 20 19:13 global_albedo4.1x1.grb
-rw-r--r--. 1 emc.glopara emc     8274 Dec 20 19:13 global_glacier.2x2.grb
-rw-r--r--. 1 emc.glopara emc     8274 Dec 20 19:13 global_maxice.2x2.grb
-rw-r--r--. 1 emc.glopara emc    39994 Dec 20 19:13 global_mxsnoalb.uariz.t126.384.190.rg.grb
-rw-r--r--. 1 emc.glopara emc   568856 Dec 20 19:13 global_o3prdlos.f77
-rw-r--r--. 1 emc.glopara emc  5468834 Dec 20 19:13 global_shdmax.0.144x0.144.grb
-rw-r--r--. 1 emc.glopara emc  5468834 Dec 20 19:13 global_shdmin.0.144x0.144.grb
-rw-r--r--. 1 emc.glopara emc    89184 Dec 20 19:13 global_slope.1x1.grb
-rw-r--r--. 1 emc.glopara emc   438768 Dec 20 19:13 global_snoclim.1.875.grb
-rw-r--r--. 1 emc.glopara emc  1919712 Dec 20 19:13 global_snowfree_albedo.bosu.t126.384.190.rg.grb
-rw-r--r--. 1 emc.glopara emc  1743072 Dec 20 19:13 global_soilmgldas.t126.384.190.grb
-rw-r--r--. 1 emc.glopara emc    21524 Dec 20 19:13 global_soiltype.statsgo.t126.384.190.rg.grb
-rw-r--r--. 1 emc.glopara emc    20094 Dec 20 19:13 global_tg3clim.2.6x1.5.grb
-rw-r--r--. 1 emc.glopara emc 15713952 Dec 20 19:13 global_vegfrac.0.144.decpercent.grb
-rw-r--r--. 1 emc.glopara emc    24602 Dec 20 19:13 global_vegtype.igbp.t126.384.190.rg.grb
-rw-r--r--. 1 emc.glopara emc   681408 Dec 20 19:13 global_zorclim.1x1.grb
drwxr-xr-x. 2 emc.glopara emc     4096 Dec 30 15:05 INPUT
-rw-r--r--. 1 emc.glopara emc     7315 Dec 20 19:13 input.nml
-rw-r--r--. 1 emc.glopara emc     1149 Dec 20 19:13 model_configure
-rw-r--r--. 1 emc.glopara emc       76 Dec 20 19:13 nems.configure
drwxr-xr-x. 2 emc.glopara emc     4096 Dec 30 15:05 RESTART
-rw-r--r--. 1 emc.glopara emc 27993768 Dec 20 19:13 RTGSST.monthly.clim.grb
-rw-r--r--. 1 emc.glopara emc    32484 Dec 20 19:13 seaice_newland.grb
-rw-r--r--. 1 emc.glopara emc    65679 Dec 20 19:13 sfc_emissivity_idx.txt
-rw-r--r--. 1 emc.glopara emc     3873 Dec 20 19:13 solarconstant_noaa_an.txt

[DusanJovic-NOAA]

Why do we need this tar file unpacked? It's only purpose is to be used as a canned case for testing the ufs-weather-model executable. Nothing else. It should be not used as a source of any configuration or input data.

[uturuncoglu]

@KateFriedman-NOAA thanks. @DusanJovic-NOAA Actually, we are copying _table files, nems.configure and also default initial conditions but i'll include _table files and nems.configure to FV3 CIME interface. The initial condition is also generated by the chgres and if we don't have any objections we could always use cghres to produce data from GFS in a desired resolution. Then, we will remove the dependency to simple-test-case/ directory.

[arunchawla-NOAA]

I am closing this issue as most things seem to be resolved here