esm_runscripts version 6.19.3 giving error message on aleph since the New Year

[tsemmler05]

Since this year, the esm_runscripts version 6.19.3 is giving an error message no matter if I want to start a completely new simulation, branch off from an existing simulation, or restarting an existing simulation to add more simulated years.

After submitting

awiiccp5@elogin1:/proj/awiiccp5:> esm_runscripts awicm3-frontiers-xios-aleph-TCO95L91-HR_generateweights.yaml --open-run -e test

I am getting this error message:

STARTING SIMULATION JOB! Experiment ID = test Setup = awicm3 This setup consists of:

• fesom
• oifs
• rnfmap
• oasis3mct

• xios Experiment is installed in: /scratch/awiiccp5//test

Traceback (most recent call last): File "/home/awiiccp5/miniconda3/bin/esm_runscripts", line 33, in sys.exit(load_entry_point('esm-tools', 'console_scripts', 'esm_runscripts')()) File "/mnt/lustre/home/awiiccp5/esm_tools/src/esm_runscripts/cli.py", line 278, in main Setup() File "/mnt/lustre/home/awiiccp5/esm_tools/src/esm_runscripts/sim_objects.py", line 69, in call self.prepcompute() File "/mnt/lustre/home/awiiccp5/esm_tools/src/esm_runscripts/sim_objects.py", line 161, in prepcompute self.config = prepcompute.run_job(self.config) File "/mnt/lustre/home/awiiccp5/esm_tools/src/esm_runscripts/prepcompute.py", line 32, in run_job config = evaluate(config, "prepcompute", "prepcompute_recipe") File "/mnt/lustre/home/awiiccp5/esm_tools/src/esm_runscripts/helpers.py", line 69, in evaluate config = esm_plugin_manager.work_through_recipe( File "/mnt/lustre/home/awiiccp5/esm_tools/src/esm_plugin_manager/esm_plugin_manager.py", line 141, in work_through_recipe config = getattr(submodule, workitem)(config) File "/mnt/lustre/home/awiiccp5/esm_tools/src/esm_runscripts/prepcompute.py", line 91, in prepare_coupler_files coupler_filename = config["general"]["coupler"].prepare( File "/mnt/lustre/home/awiiccp5/esm_tools/src/esm_runscripts/coupler.py", line 56, in prepare self.add_couplings(full_config) File "/mnt/lustre/home/awiiccp5/esm_tools/src/esm_runscripts/coupler.py", line 370, in add_couplings self.coupler.add_coupling( File "/mnt/lustre/home/awiiccp5/esm_tools/src/esm_runscripts/oasis.py", line 128, in add_coupling alltimes = transformation.get("time_transformation", "bla") AttributeError: 'NoneType' object has no attribute 'get'

This happens no matter which (previously) functioning run-script I would submit from this directory.

It is taking the esm_tools version from /home/awiiccp5/esm_tools and the esm_runscripts executable from /home/awiiccp5/miniconda3/bin

[mandresm]

Hi @tsemmler05!

I had a first look at this and doing a git diff in your esm_tools I noticed the following:

cd /home/awiiccp5/esm_tools
git diff configs/setups/awicm3/awicm3.yaml

diff --git a/configs/setups/awicm3/awicm3.yaml b/configs/setups/awicm3/awicm3.yaml
index 65e11a8e..404c2f5a 100644
--- a/configs/setups/awicm3/awicm3.yaml
+++ b/configs/setups/awicm3/awicm3.yaml
@@ -583,7 +583,7 @@ oasis3mct:
                 rstas.nc:
                         - 'heat_oce:heat_swo <--gauswgt_gss-- A_Qns_oce:A_Qs_all'
                         - 'prec_oce:snow_oce:evap_oce:subl_oce <--gauswgt_gss-- A_Precip_liquid:A_Precip_solid:A_Evap:A_Subl'
-                        - 'heat_ico <--gauswgt_gss-- A_Q_ice'
+                        - 'heat_ico <--gauswgt_glo-- A_Q_ice'
                         - 'taux_oce:tauy_oce:taux_ico:tauy_ico <--bicubic-- A_TauX_oce:A_TauY_oce:A_TauX_ice:A_TauY_ice'
                         - 'R_Runoff_atm:R_Calving_atm <--gauswgt_gss-- A_Runoff:A_Calving'
                         - 'hydr_oce:enth_oce <--gauswgt_gss-- R_Runoff_oce:R_Calving_oce'

You have changed the mapping method in AWICM3 for A_Q_ice to heat_ico. However, I cannot find that remapping method described neither in your runscript or in configs:

$ grep -r gauswgt_glo configs

configs/setups/foci/foci.yaml:                              - 'OTotRain <--gauswgt_global-- ATotRain'
configs/setups/foci/foci.yaml:                              - 'OTotRain <--gauswgt_global-- ATotRain'
configs/setups/foci/foci.yaml:                gauswgt_global:
configs/setups/awicm3/awicm3.yaml:                        - 'heat_ico <--gauswgt_glo-- A_Q_ice'

Any method you ask to use in rstas.nc or rstos.nc needs to be described in one of the yaml files, under the coupling_methods dictionary, as you are currently doing in your runscript:

# Your runscript awicm3-frontiers-xios-aleph-TCO95L91-HR_generateweights.yaml
general:
    setup_name: "awicm3"
    version: "frontiers-xios"
    account: "awiiccp"
    compute_time: "02:10:00"
    initial_date: "1850-01-01"
    final_date: "1850-01-02"
    [ ... ]

oasis3mct:
    lresume: false # Set to false to generate the rst files for first leg
    time_step: 3600
    ini_parent_dir: /proj/awi/input/oasis/cy43r3/TCO95-HR/
    ini_restart_dir: /proj/awi/input/oasis/cy43r3/TCO95-HR/
    input_dir: /proj/awi/input/oasis/cy43r3/TCO95-HR/
    coupling_methods:
        # there is no gauswgt_glo here
        gauswgt_c:
            time_transformation: conserv
            remapping:
                gauswgt:
                    search_bin: latitude
                    nb_of_search_bins: 1
                    nb_of_neighbours: 9
                    weight: "2"
            postprocessing:
                conserv:
                    method: glbpos
                    algorithm: opt
        gauswgt_gss:
            time_transformation: conserv
            remapping:
                gauswgt:
                    search_bin: latitude
                    nb_of_search_bins: 1
                    nb_of_neighbours: 9
                    weight: "2"
            postprocessing:
                conserv:
                    method: gsspos
                    algorithm: opt
        gauswgt_i:
            time_transformation: instant
            remapping:
                gauswgt:
                    search_bin: latitude
                    nb_of_search_bins: 1
                    nb_of_neighbours: 9
                    weight: "2"

Let me know if after fixing this problem things start to work again.

Kind regards, Miguel

[tsemmler05]

That worked! Thanks for spotting this, @mandresm. I indeed had to change the interpolation method for the deep future run and had forgotten about it. It would be nice if the esm_tools could give a more indicative error message.

[mandresm]

It would be nice if the esm_tools could give a more indicative error message.

Indeed. We'll work in a better error handling.

[mandresm]

Hi @tsemmler05,

You should get a nice error message now if this problem ever happens again, given that you have the latest version of ESM-Tools. See #1121

Kind regards, Miguel