Setup script does not stage sfcanal_data.tile warm-start ICs

[TravisElless-NOAA]

What is wrong?

When using --icsdir to stage warm-start initial conditions as part of setup_expt.py, the sfcanl_data.tile files located in the CDATE model_data/atmos/restart/ directory are not being staged in the comroot.

What should have happened?

All files needed to warm-start an experiment should be gathered by setup_expt.py

What machines are impacted?

All or N/A

Steps to reproduce

1. Run setup_expt.py for a warm-start experiment using the --icsdir option.
2. Check what files are present in the comroot.

Additional information

Example output for a 3DVAR test from running this setup script is located at /scratch2/NCEPDEV/stmp3/Travis.J.Elless/comrot/test_setup using ICs from /scratch2/NCEPDEV/stmp3/Travis.J.Elless/comrot/test_warm_start/ and --idate 2021080500

Do you have a proposed solution?

No response

[aerorahul]

sfc_data.tileX.nc is the warmstart file. sfcanl_data.tileX.nc is the analysis tile file (not a warm start). Your best bet is to stage the sfcanl_data.tileX.nc replacing the sfc_data.tileX.nc and then execute setup_expt.py.

[TravisElless-NOAA]

Given that these files are included in the restarta files and setup_expt.py grabs other analysis files from this restart group, shouldn't it be grabbing the analysis surface files as well? Without these files, we lose reproducibility when restarting simulations.

[aerorahul]

Yes. We should. The situation you are describing is an identical start from a previous experiment. We did not have a requirement such as that. We can add a feature request for it in the future.

[aerorahul]

We are working on a refactoring of the setup_expt.py that will drive the files one needs from a yaml.
It will then be up to the user to describe the files they want to copy from/to.

[WalterKolczynski-NOAA]

@KateFriedman-NOAA please confirm this has been solved.

[KateFriedman-NOAA]

This has been resolved. The updated staging job now includes the sfcanl_data.tile files if they exist in the ICSDIR:

{% if path_exists(ICSDIR ~ "/" ~ COMOUT_ATMOS_RESTART_PREV_MEM | relpath(ROTDIR) ~ "/" ~ p_prefix ~ ".sfcanl_data.tile" ~ ntile ~ ".nc") %}
- ["{{ ICSDIR }}/{{ COMOUT_ATMOS_RESTART_PREV_MEM | relpath(ROTDIR) }}/{{ p_prefix }}.sfcanl_data.tile{{ ntile }}.nc", "{{ COMOUT_ATMOS_RESTART_PREV_MEM }}"]
{% endif %} # path_exists

https://github.com/NOAA-EMC/global-workflow/blob/develop/parm/stage/atmosphere_warm.yaml.j2#L24:L26

Closing this as complete.