on-the-fly preprocessing?

[perrette]

add an option to add a custom function to preprocess climate variables before inputting it into rime preprocessing?

[first posted in https://github.com/iiasa/rime/issues/23#issue-2473505220]

[NiklasSchwind]

The thought behind this idea is to save some disk space for the user by not having to first process the default indicators into the indicator you want to emulate (e.g. daily tasmax into yearly maximum tasmax if you want to emulate yearly maximum tasmax) and then apply the package but that the package can include such processing steps (and thus emulate yearly maximum tasmax from daily tasmax in our example). Including the preprocessing of the variable in the package would omit the need to save yearly maximum tasmax simulations as an in-between step.

However, I would say it is a low priority and I wouldn't implement it if it becomes complicated or makes the code ugly.

[perrette]

Hi Niklas, I am not sure I follow you 100%. In my mind I understood this issue as adding an option to do some of the work behind the scene (like data download and preprocessing), instead of explicitly executing all these steps with dedicated commands. The way I figured it would not save any disk space compared to doing it directly. But we can discuss this elsewhere possibly. Anyway, I'll mark it as low priority and possibly out of this milestone, right? (more as an unspecified rimeX issue)

[NiklasSchwind]

Your interpretation might be a good idea to explore as well! However, I meant to enable users to write their own preprocessing function that e.g. preprocesses daily tasmax to TXx and includes it in the configuration so that rimeX automatically applies this function to the tasmax data to produce the TXx data, then produces the records for TXx and afterward automatically delete the TXx data again. But, I think we can just not do this, maybe it complicates things too much. The alternative is to preprocess tasmax first into TXx outside of rimeX and then apply rimeX to TXx. This is anyway a low priority adjustment imo.

[perrette]

We'll see clearer when we have built-in preprocessing for this in the first place.