caracal-pipeline / stimela

Stimela 2.0
GNU General Public License v2.0
5 stars 3 forks source link

it would be nice to have a for-loop recipe output that is a list of per-iteration outputs from a particular step #294

Open o-smirnov opened 4 months ago

o-smirnov commented 4 months ago

Quite possibly this can be done via the current aliasing mechanism. As things stand, the current aliased outputs will return the output value from the last iteration. But a flag could be provided in the schema to change that.

o-smirnov commented 4 months ago

The question is, should it be an explicit flag, or some magic syntax in the aliases specification? E.g.:

  outputs:
    loop-list-output:
      aliases: [step.output]
      collect_list_output: true

vs

  outputs:
    loop-list-output:
      aliases: [step.output@]

The latter is somewhat more esoteric, but has the advantage of also being compatible with the standalone aliases section.

o-smirnov commented 4 months ago

Upon some thinking -- there is little point in having a for-loop output that corresponds to the output of one particular iteration (and this esoteric use case could anyway better be accomplished with formula syntax), so perhaps any step output alias defined in a for-loop should implicitly become a List[dtype] recipe output.