TonyB9000
commented
8 months ago I was able to fix most of the "mon versus day" issues by adding these lines to main.py: _run_info_mode()
if self.freq == "mon" and handler['table'] == "CMIP6_day.json":
continue
if ( self.freq == "day" or self.freq == "3hr" ) and handler['table'] == "CMIP6_Amon.json":
continueHowever, this fails to fix the issue with rsut" andrsutcs`, as they both are "Amon" table entries:
TASK = Amon_rsutcs.yaml : COMMAND = e3sm_to_cmip --info -v rsutcs --freq mon --realm atm -t $TABLE_PATH --map no_map --info-out yaml/Amon_rsutcs.yaml
- CMIP6 Name: rsutcs
CMIP6 Table: CMIP6_Amon.json
CMIP6 Units: W m-2
E3SM Variables: SOLIN, FSNTOAC [What’s going on here?]
- CMIP6 Name: rsutcs
CMIP6 Table: CMIP6_Amon.json
CMIP6 Units: W m-2
E3SM Variables: FSUTOAC [What’s going on here?]
TASK = Amon_rsut.yaml : COMMAND = e3sm_to_cmip --info -v rsut --freq mon --realm atm -t $TABLE_PATH --map no_map --info-out yaml/Amon_rsut.yaml
- CMIP6 Name: rsut
CMIP6 Table: CMIP6_Amon.json
CMIP6 Units: W m-2
E3SM Variables: SOLIN, FSNTOA [What’s going on here?]
- CMIP6 Name: rsut
CMIP6 Table: CMIP6_Amon.json
CMIP6 Units: W m-2
E3SM Variables: FSUTOA [What’s going on here?]In addition - two new issues were created, because the ONLY handler for (day) huss is/was the "Amon" handler, which is now eliminated as "Amon" does not match "day", Same issue with (day)tas. It appears that this "overload" was tolerated because both the "day" and "mon" versions for huss and tas use the same formula. But without formal indirection, it is bad form to rely upon "Hey, just one handler seems to match - it must be the right one." Instead, we must create TWO handlers with distinct names, even if their content is identical.
What happened?
I wrote a test script that takes every CMIP6 dataset_id we have defined in the dataset_spec, isolates the “table.var” portion (129 unique pairs exist), derives the “E2C-required” freq and realm from the “table” name, and issues a (mode 2) info request for each unique pair.
In 93% of cases (120/129), a single unambiguous specification is returned. But the 9 ambiguous cases must be corrected in E2C, which will be my next task. These are the following:
(NOTE: in each case, TABLE_PATH = /p/user_pub/e3sm/staging/resource/cmor/cmip6-cmor-tables/Tables)
The first order of business will be to fix e2c so that the ONLY structure returned corresponds to the frequency requested (ONE, not TWO).
Also, according to https://acme-climate.atlassian.net/wiki/spaces/DOC/pages/925304036/Amon+variable+conversion+table,
Forcing the correct outputs in these 9 cases should address issue #
What did you expect to happen? Are there are possible answers you came across?
Single yaml item output for requested frequency. For instance: With
COMMAND = e3sm_to_cmip --info -v pr --freq day --realm atm -t $TABLE_PATH --map no_map --info-out yaml/day_pr.yamlwe expect:
not
Minimal Complete Verifiable Example (MVCE)
Relevant log output
Anything else we need to know?
No response
Environment
populated config files : /home/bartoletti1/mambaforge/.condarc conda version : 24.1.2 conda-build version : not installed python version : 3.10.6.final.0 solver : libmamba (default) virtual packages : archspec=1=broadwell conda=24.1.2=0 glibc=2.17=0 linux=3.10.0=0 __unix=0=0 base environment : /home/bartoletti1/mambaforge (writable) conda av data dir : /home/bartoletti1/mambaforge/etc/conda conda av metadata url : None channel URLs : https://conda.anaconda.org/conda-forge/linux-64 https://conda.anaconda.org/conda-forge/noarch package cache : /home/bartoletti1/mambaforge/pkgs /home/bartoletti1/.conda/pkgs envs directories : /home/bartoletti1/mambaforge/envs /home/bartoletti1/.conda/envs platform : linux-64 user-agent : conda/24.1.2 requests/2.31.0 CPython/3.10.6 Linux/3.10.0-1160.108.1.el7.x86_64 rhel/7.9 glibc/2.17 solver/libmamba conda-libmamba-solver/24.1.0 libmambapy/1.5.7 UID:GID : 61843:4061 netrc file : None offline mode : False