Variables table updating

[ljoakim]

Fixes WCRP-CORDEX/cordex-cmip6-cmor-tables#22

I've made a first attempt to update the variables table from the github cmor tables. I took a slightly different approach than I originally planned: instead of generating a new table, the current table is read and updated. This means there should already be a row for each variable, with all required columns filled out, except for the columns that are updated from the cmor tables.

Currently only the following columns are updated:

• Cell Method (subdaily,CORDEX) from cell_methods of either 1hr or 6hr table
• Cell Method (daily,CORDEX) from cell_methods of day table
• Cell Method (monthly,CORDEX) from cell_methods of mon table
• units, long_name and standard_name from day or fx table, depending on where the variable is found.

I've updated the table using the update script, please look at the updated table in the PR.

1. Are there more columns that should be updated?
2. Should fx variables have a value for cell_methods? (unsure where to put it in the table)

[gnikulin]

• I think it's the best solution to update units, long_name, standard_name and cell_methods, these 4 attributes are the minimum we need.
• We can also add/update comment, not mandatory but good to have as additional explanations for a number of variables, although it's not clear for me where to put commentin the table @doblerone ?
• we do need cell_methods for the fx variables as defined in CORDEX-CMIP6_fx.json

[ljoakim]

Note: I've changed the third column of the cape row from CAPE to cape manually.

[ljoakim]

Unfortunately the updated cell method strings in the cmor tables, when inserted into the variable table in this repo, breaks the logic in cmorlight.py, causing an exception. I'm looking into it.

[ljoakim]

Regarding the cell_method problem, there are three functions dealing with this: proc_seasonal, process_file and process_file_fix. The first one, for seasonal (sem), I don't think I have any data to test with. Will there ever be any seasonal data to cmorize?

[gnikulin]

Seasonal means were only defined in CORDEX-CMIP5 and were useless, no seasonal means in CORDEX-CMIP6. In general, we can delete all parts of the code related to the seasonal mean output.

[ljoakim]

• I've added the fx cell methods to the column Cell Method (additional subdaily,CORDEX). It does not seem to be used by any other types, so should be fine.
• I fixed the error when using the new cell_methods strings. The implementation now assumes that in the cmor tables, for all variables that are not fx, the cell_methods string conatins the substring time:. The substring after the first occurence of time: is what is passed on to the 'old' code (see implementation).

[ljoakim]

...and comment is now also added to the table, and to the variable in the output file.

[ljoakim]

I've removed the draft status, so if you think all is ok (with the comments I've added above), this PR should be ready for merge. I'm running a few additional tests, but if anything pops up, we can handle it in a separate issue.

[gnikulin]

Great !!!! A quick look at the var table shows that all changes/fixes should be fine. We can merge this PR and then fix "residual" problems if any.

[ljoakim]

I found one additional error for day/prhmax, where the cell_methods again caused the error. I think I need to discuss which operation to run for which cases. E.g.:

• day/prhmax has "cell_methods": "area: mean time: mean within hours time: maximum over hours". I assume maximum (over hours) is the operation when producing day output? (the operation is used when calling cdo)
• mon/tasmax has "cell_methods": "area: mean time: maximum within days time: mean over days". Here mean (over days) should be used right?

Probably the last occurrence of time: should be used to find the operation, not the first occurrence which is currently used.

[ljoakim]

Probably the last occurrence of time: should be used to find the operation, not the first occurrence which is currently used.

Correction: studying the code and previous behaviour, I think it is correct to use the first occurence.

So, I believe the code as it is now is working for all variables, except for prhmax due to it's particular cell_method. I suggest we open a separate issue for that, to get this PR merged. Then we can start cmorizing other variables for analysis/validation. Would that be ok?

[gnikulin]

Yes, we'll wait until this issue with monthly prhmaxis clarified, https://github.com/WCRP-CORDEX/discuss/issues/7