scipp / esssans

SANS data reduction for the European Spallation Source
https://scipp.github.io/esssans/
BSD 3-Clause "New" or "Revised" License
0 stars 2 forks source link

Restructure workflow to facilitate workflow partitioning for efficient streaming execution #163

Closed SimonHeybrock closed 1 month ago

SimonHeybrock commented 2 months ago

There are two structure changes here:

  1. Split the wavelength-reduction (giving the new ReducedQ) from the normalization. This is essential for streamed processing since we need to accumulate data (numerator and denominator) before normalization, but as late as possible in the workflow. The wavelength-reduction is relatively expensive since it performs binned data operations, e.g., to handle wavelength bands. Further, accumulating event data must be avoided since it will quickly run out of memory.
  2. Split the detector term from the monitor term in the computation of the denominator. As the detector-term (solid angle and direct beam function) is static, it can be pre-computed. This is important since it involves a relatively costly reduction over many pixels in Q-bins.