search

ACCORD-NWP / DAVAI-tests

DAVAÏ tests templates and config files
Other
0 stars 8 forks source link

Comparison issue with the Canonical forecasts on ATOS #25

Open romick-knmi opened 1 month ago

romick-knmi commented 1 month ago

In testing on ATOS, it was noticed that a comparison issue arises from the canonical_forecasts.py job if it has already been run. It happens in the CanonicalArpegeAplForecast(tag='forecast-arpege_apl-global798c22', ticket=t, **kw), task node. A current workaround is to delete the canonical forecast directories within the cache of mtool, e.g. rm -rf /scratch/nlcr/mtool/cache/vortex/davai/nrv/dv-0051-atos_bologna@nlcr/20200818T0000A/canonical-forecast-arpege*-global798c22.ompiifc2302*.x. Then the comparison issue is corrected.

The current idea of the cause of this issue is from tasks/forecasts/canonical/arpege_apl.py, where at line 50 is the following : 

       # 1.1.0/ Reference resources, to be compared to:
        if 'early-fetch' in self.steps or 'fetch' in self.steps:
            self._wrapped_input(**self._reference_continuity_expertise())
            self._wrapped_input(**self._reference_continuity_listing())
            # this task is to be compared to another task of the same experiment
            self._wrapped_input(**self._reference_consistency_expertise())
            self._wrapped_input(**self._reference_consistency_listing())

This appears to be a comparison within the same davai experiment.

Thus, the directories may need to be cleaned or an updated file checked.

AlexandreMary commented 1 month ago

OK, that is an issue for which I have a fix ready, I think: in this test the continuity (i.e. same task but reference experiment) outputs to compare to, and the consistency (i.e. a different task from the same experiment) outputs, have the same name, and the comparison get confused. Will push the fix asap.