European-XFEL / karabo_data

Python tools to read and analyse data from European XFEL
https://karabo-data.readthedocs.io/
BSD 3-Clause "New" or "Revised" License
13 stars 7 forks source link

Assemble detector modules without copying them into a 'stack' array #188

Closed takluyver closed 5 years ago

takluyver commented 5 years ago

From discussion relating to KaraboFAI: if you want to load data from AGIPD/LPD/DSSC and assemble it, it currently gets copied twice in memory:

individual module arrays -> stacked array -> assembled image

The first of these copies should be unnecessary. There are two basic schemes for how we could avoid it:

zhujun98 commented 5 years ago

I would go for the first method and make a StackedData class which has several methods to provide the necessary np.array interface, e.g. shape property:

  1. The stacked data provides the similar interface as the data from the calibration pipeline, although one needs to move the axis twice for LPD and DSSC. Therefore, it is good to 'keep' it;
  2. Depending on the level of backward compatibility you want to have, there is a few options to make the new stack_detector_data function. No matter what, you do not need to touch the method position_modules_fast which is used for assembling.
takluyver commented 5 years ago

PR #196 is an initial attempt at implementing option 1.

takluyver commented 5 years ago

Done in #196.