Closed jeromebarre closed 1 week ago
@jeromebarre Try again please.
@jeromebarre I don't know how realistic this is until the Skylab v4 release. Our current plan is to be able to build and run unit tests for ATM, NG-GODAS and S2S, in addition to running the UFS ATM in Skylab v4 via EWOK. This is already a stretch, and given the complexity of getting all the input data in place and compiling the UFS with the aerosol options turned on, I'd say this is more realistic for Skylab v5.
How can we help you?
The current set of PRs compiles the UFS ATM, NG-GODAS and S2S applications on macOS. It also compiles the ATM on the AWS Parallelcluster, but NG-GODAS and S2S fail with a linker error (issues with fPIC, static libraries etc). This needs to be fixed first.
Second, we have working ATM ctests, which Clementine is using as blueprint for setting up the EWOK experiments. The existing NG-GODAS ctests are broken since we updated MOM6, CICE6 etc. to stay in sync with the ufs-weather-model. These need to be fixed.
Third, we need a set of input data (a run directory will do) for the S2S configuration and get those into the ctests.
Fourth, we need to add the S2SA or S2SWA or whatever the exact aerosol configuration (application in ufs-weather-model lingo) is that you need, and add a ctest for those. This requires fixing a few build errors related to GOCART-MAPL-ESMF. See https://github.com/ufs-community/ufs-weather-model/issues/1498 for more information.
And this is the issue that needs to be addressed ASAP, before/in parallel with items 1-3 above: https://github.com/jcsda/ufs-bundle/issues/19
Just to recap on our recent conversations @climbfuji . ATMAERO is needed not S2SWA. However S2SWA would be needed in the future if we plan to run aerosol with the coupled model. For now ATMAERO is the priority for composition.
@jeromebarre I have a draft PR that I am testing with the other UFS configurations on my Mac: https://github.com/JCSDA/ufs-bundle/pull/21. This will address step one of this issue, building the UFS APP ATMAERO in ufs-bundle.
Great! Thx for looking into this.
https://github.com/JCSDA/ufs-bundle/pull/21 was merged. Next step is to add a ctest for atmaero (ideally using the existing UFS c48 warmstart ICs, just adding the additional data needed for the aerosols? Is this something that @jeromebarre and/or @mer-a-o can do?
I can do that. If a warmstart and then IC files are needed, I need to get the info from @SarahLu-NOAA that managed to run a UFS run with the NOAA workflow. I will try to get this info.
I maybe be able to help with getting those files if needed.
Yes that would be great. We'd need emission files and IC files. Let me check with @SarahLu-NOAA first.
At Orion, CEDS emissions arch covers 2011-2020 and QFED archive covers 2011-2019.
I test run (UFSAero, cold started from met-only GDAS) is for 2019-08-01.
I'm in the process of reactivating my orion account, so can't access my note at Orion at this moment.
@jeromebarre @climbfuji GOCART emissions files can be found at: HERA: /scratch1/NCEPDEV/global/glopara/data/gocart_emissions ORION: /work2/noaa/global/wkolczyn/noscrub/global-workflow/gocart_emissions S4: /data/prod/glopara/gocart_emissions
The path to the input data (AERO_INPUTS_DIR) is specified in global-workflow/parm/config/config.aero
NOTE QFED emissions cover 2011-2019 and CEDS emissions cover 2011-2020.
I did one ATMAERO cold-start test run (SDATE=20190801) at Orion mid Feb and Hera this week.
The ICs are fetched from HPSS: /NCEPPROD/hpssprod/runhistory/rh2019/201908/20190801/gpfs_dell1_nco_ops_com_gfs_prod_gfs.20190801_00.gfs_nemsioa.tar
You can copy gfs.t00z.atmanl.nemsio and gfs.t00z.sfcanl.nemsio from HERA: /scratch1/NCEPDEV/global/Sarah.Lu/save/UFS/ICs/gfs.20190801. This is v15 ICs, so global_workflow call this routine to prepare ICs: global-workflow/sorc/ufs_utils.fd/util/gdas_init/run_v15.chgres.gfs.sh
My test run uses QFED emissions. AERO_EMIS_FIRE=QFED, specified in global-workflow/parm/config/config.aero
Once QFED is selected, UFS/global-workflow/parm/chem/ExtData.qfed will specify the emissions files for SU, OC, BC, and NH3 global-workflow/parm/chem directory contain resource parameters for running GOCART
The global_workflow link input data and copy config files for GOCART during run-time. @climbfuji Ping me if you need additional information or have any questions. I can test your workflow at HERA, S4, or Cheyenne. I can't access ORION at this moment.
@SarahLu-NOAA thanks! with that I think I will be able to set up a test. Likely next week.
@climbfuji @mer-a-o Dom, For your EWOK, where are fixed files directories (fix/am, fix/orog ... etc) are located at S4, Orion, and Cheyenne? Thanks.
See https://jointcenterforsatellitedataassimilation-jedi-docs.readthedocs-hosted.com/en/latest/using/building_and_running/running_skylab.html#create-and-source-jedi-root-activate-sh please. In the script activate.sh
, an environment variable EWOK_STATIC_DATA
is set, dependent on the machine.
Thanks, Dom. @mer-a-o From activate.sh, the fixed files at s4 can be found at /data/prod/jedi/static/skylab-4.0.0/ufs For instance, /data/prod/jedi/static/skylab-4.0.0/ufs/c384/global_snoclim.1.875.grb is identical to /data/users/clu/UFS/fix/am/global_snoclim.1.875.grb
@mer-a-o To run aerosols, ExtData directory needs to be copied to the run directory ExtData can be found at S4: /data/prod/glopara/gocart_emissions
@mer-a-o @jeromebarre will start with reviewing Maryam's note on her efforts
@gibbsp @symoore90 please move this to aop23
Build, compile and execute a version UFS with aerosol (GOCART or GOCART2) and chemistry (RAQMS or else).
Steps:
since I cannot put assignees right now I tag people here: @SarahLu-NOAA @jeromebarre @climbfuji