forcing feedback on AM5 test run

[aradhakrishnanGFDL]

Here is the error I get while running forcing feedback POD on GFDL's dpdev, from within the MDTF container. Please refer to config files and catalogs used in this repository https://github.com/aradhakrishnanGFDL/testing.

runtime configuration https://github.com/aradhakrishnanGFDL/testing/blob/main/runtime_config_am5test.jsonc

mdtf_framework -f runtime_config_am5test.jsonc

POD convention and data convention are both cmip. No data translation will be performed for case atmos_cmip. Preprocessing data for forcing_feedback Querying /proj/wkdir/c96L65_am5f7b10r0_amip30.json for variable ts for case atmos_cmip. WARNING: /opt/conda/envs/_MDTF_base/lib/python3.12/site-packages/intake_esm/_search.py:50: UserWarning: This pattern is interpreted as a regular expression, and has match groups. To actually get the groups, use str.extract. mask = df[column].str.contains(value, regex=True, case=True, flags=0)

CRITICAL: ** Uncaught exception: Traceback (most recent call last): File "/proj/MDTF-diagnostics/mdtf_framework.py", line 243, in exit_code = main(prog_name='MDTF-diagnostics') ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/conda/envs/_MDTF_base/lib/python3.12/site-packages/click/core.py", line 1157, in call return self.main(args, kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/conda/envs/_MDTF_base/lib/python3.12/site-packages/click/core.py", line 1078, in main rv = self.invoke(ctx) ^^^^^^^^^^^^^^^^ File "/opt/conda/envs/_MDTF_base/lib/python3.12/site-packages/click/core.py", line 1434, in invoke return ctx.invoke(self.callback, ctx.params) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/conda/envs/_MDTF_base/lib/python3.12/site-packages/click/core.py", line 783, in invoke return __callback(args, *kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/conda/envs/_MDTF_base/lib/python3.12/site-packages/click/decorators.py", line 33, in new_func return f(get_current_context(), args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/proj/MDTF-diagnostics/mdtf_framework.py", line 199, in main cat_subset = data_pp.process(cases, ctx.config, model_paths.MODEL_WORK_DIR) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/proj/MDTF-diagnostics/src/preprocessor.py", line 1314, in process cat_subset = self.query_catalog(case_list, config.DATA_CATALOG) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/proj/MDTF-diagnostics/src/preprocessor.py", line 1005, in query_catalog if var_xr.get(att, None) is not None: ^^^^^^^^^^ AttributeError: 'list' object has no attribute 'get'

[jtmims]

@aradhakrishnanGFDL I've added a simple logic statement to get over the hurdle you're noticing. I've noticed that it still gets hung up after this. For the catalog query, the framework is searching for standard_name 'surface_downwelling_shortwave_flux_in_air' for the variable 'rsds'. In your catalog, the standard_name is supplied as 'downwelling_shortwave_flux_in_sea_water' for 'rsds'.

[aradhakrishnanGFDL]

thanks @jtmims. I updated my branch to main, but I see the same error again. Any thoughts?

Preprocessing data for forcing_feedback
Querying /proj/wkdir/c96L65_am5f7b10r0_amip30.json for variable ts for case atmos_cmip.
WARNING: /opt/conda/envs/_MDTF_base/lib/python3.12/site-packages/intake_esm/_search.py:50: UserWarning: This pattern is interpreted as a regular expression, and has match groups. To actually get the groups, use str.extract.
  mask = df[column].str.contains(value, regex=True, case=True, flags=0)

CRITICAL: **********************************************************************
Uncaught exception:
Traceback (most recent call last):
  File "/proj/MDTF-diagnostics/mdtf_framework.py", line 243, in <module>
    exit_code = main(prog_name='MDTF-diagnostics')
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/conda/envs/_MDTF_base/lib/python3.12/site-packages/click/core.py", line 1157, in __call__
    return self.main(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/conda/envs/_MDTF_base/lib/python3.12/site-packages/click/core.py", line 1078, in main
    rv = self.invoke(ctx)
         ^^^^^^^^^^^^^^^^
  File "/opt/conda/envs/_MDTF_base/lib/python3.12/site-packages/click/core.py", line 1434, in invoke
    return ctx.invoke(self.callback, **ctx.params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/conda/envs/_MDTF_base/lib/python3.12/site-packages/click/core.py", line 783, in invoke
    return __callback(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/conda/envs/_MDTF_base/lib/python3.12/site-packages/click/decorators.py", line 33, in new_func
    return f(get_current_context(), *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/proj/MDTF-diagnostics/mdtf_framework.py", line 199, in main
    cat_subset = data_pp.process(cases, ctx.config, model_paths.MODEL_WORK_DIR)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/proj/MDTF-diagnostics/src/preprocessor.py", line 1316, in process
    cat_subset = self.query_catalog(case_list, config.DATA_CATALOG)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/proj/MDTF-diagnostics/src/preprocessor.py", line 1007, in query_catalog
    if var_xr.get(att, None) is not None:
       ^^^^^^^^^^
AttributeError: 'list' object has no attribute 'get'

(Separate issue- standard_name issue noted, useful tip. The issue is because the same variable_id/name does not have unique standard names, hence I have to use realm. This will be taken care of in catalog builder)

[jtmims]

@aradhakrishnanGFDL I apologize! I found 3 problems, and I completely forgot to mention the first one. That is on me. In the runtime config file, you currently have the date in 'yyyymmdd' format. The framework is not the biggest fan of this format for some monthly PODs (especially NCL PODs). This is something we will most likely have to look more into, but in the meantime you can just use the 'yyyymm' format. In the config, this would look like:

"case_list":
    {
      "atmos_cmip":
        {
          "model": "am5",
          "convention": "CMIP",
          "startdate": "198001",
          "enddate": "198012"
        }
    },

In my testing, this was what got the framework past the error message you are currently seeing.

[aradhakrishnanGFDL]

Excellent, that helped. Thanks! Closing the issue now.