Unknown input variables mmrprcpoxn, mmrprcpoxs, mmrprcprdn

[lewisblake]

Often times when running an analysis, or even just running pytest, exceptions are raised for unknown input variables.

------------------------------------------------------------------------- Captured stdout call -------------------------------------------------------------------------
Unknown input variable mmrprcpoxn
Unknown input variable mmrprcpoxs
Unknown input variable mmrprcprdn
------------------------------------------------------------------------- Captured stderr call -------------------------------------------------------------------------
--- Logging error ---
Traceback (most recent call last):
  File "/lustre/storeB/users/lewisb/Python/Evaluations/pyaerocom/pyaerocom/variable.py", line 509, in parse_from_ini
    var_name = self._check_aliases(var_name)
  File "/lustre/storeB/users/lewisb/Python/Evaluations/pyaerocom/pyaerocom/variable.py", line 409, in _check_aliases
    return _check_alias_family(var_name, ap)
  File "/lustre/storeB/users/lewisb/Python/Evaluations/pyaerocom/pyaerocom/variable_helpers.py", line 86, in _check_alias_family
    raise VariableDefinitionError(
pyaerocom.exceptions.VariableDefinitionError: Input variable could not be identified as belonging to either of the available alias variable families

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/lewisb/miniconda3/envs/pya/lib/python3.10/logging/handlers.py", line 74, in emit
    self.doRollover()
  File "/home/lewisb/miniconda3/envs/pya/lib/python3.10/logging/handlers.py", line 432, in doRollover
    if self.backupCount > 0:
TypeError: '>' not supported between instances of 'str' and 'int'

....

Any ideas what might be causing this? I can not find these variables in the code

[lewisblake]

Check Colocator.prepare_run()

[lewisblake]

Happens in Colocator.prepare_run() when running Colocator._find_var_matches()

[lewisblake]

Example test for debugging https://github.com/metno/pyaerocom/blob/8a1a4a16466e82ead958194764e493c0422adef7/tests/test_colocation_auto.py#L121

[lewisblake]

For record keeping: It happens here

https://github.com/metno/pyaerocom/blob/8a1a4a16466e82ead958194764e493c0422adef7/pyaerocom/io/readgridded.py#L504

when checking crearted-on-the-fly mmr* vars, which need not actually exist. It's coming from a logger.info() call

https://github.com/metno/pyaerocom/blob/8a1a4a16466e82ead958194764e493c0422adef7/pyaerocom/variable.py#L510

[lewisblake]

999 take this output and puts it in logs, so maybe not so much of an issue for now.

[thorbjoernl]

I think the issue is in the pattern matching logic of ReadGridded._check_var_match_pattern() function, not in Variable. The pattern matching produces a variable name which does not exist in aliases.ini or variables.ini, which then produces a valid message in variable.py.

A useful breakpoint for further debugging this issue is placing a conditional breakpoint with var_name == 'concprcpoxn' and "*" in addvar in line 480 of readgridded.py. This places you at the start of a run where pattern-matching results in var_name == 'mmrprcpoxn' which is non-existent. Use this while debugging [False-monthly-daily] of test_colocator_model_ts_type_read