Generalized mask: apply a mask built from a dataset to all other datasets

[valeriupredoi]

Hi @alistairsellar @eleanorgb @ehogan - this is a follow-up to our discussion we had at the MO during the ESMValTool workshop (my apologies for the slightly tardy time to open this, been busy with the release that is now done).

Let me try and recap in broad lines what the requirements of the functionality are, but by all means, please add to this in here so we can get the full specifications together, so I can do the implementation dance :smiley: :+1:

• a mask can be built from an existing dataset (in the datasets list in the recipe, whether it be model data or OBS/obs4mips etc data)
• generation of that mask should be based on user input in the form of kwargs of the mask preprocessor
• that mask should be applied to all datasets that need to run in a diagnostic

I have started working on a prototype in #1803 - that does the following:

• a preprocessor called mask_generalized ingests two cubes - one to be masked with, the other that the mask will be built from, and a dictionary holding the masking operation - first use case here is above threshold masking; then it builds the mask from mask_cube and applies it to the generic cube that denotes all other datasets; you can see this prototype tested in its unit test. The actual API implementation and frontend via recipe is to follow, but I thought I'd first touch base with you to draft the specifications of the backend, namely:

• am I going down the right path?

• could you please provide me with a few actual real world use cases that I can implement?

BTW am CC-ing @bouweandela and @schlunma (will possibly CC the rest of the dev community as well, in the near future, after a few preliminary tosses of ideas among us)

Cheers :beer:

[valeriupredoi]

@alistairsellar do you think this is still something we should look at? It's been exactly one year since we talked about it, and I've not heard anything since :cry:

[alistairsellar]

Hi @valeriupredoi sorry for the delay - the original issue flew by without me spotting it.

Just had a chat with @eleanorgb and we think that your prototype is headed down the right path (and looks like it will be useful more widely than permafrost) but we need to refresh on exactly how @eleanorgb's personal code handles a wider range of CMIP models. Unfortunately there are several special cases for diagnosing the land ice mask and the code now in ESMValTool only handles a small number. Hence need to refresh on those and document the cases in order to give clear requirements. We are planning to sit down together in January to do that.