search

European-XFEL / karabo_data

Python tools to read and analyse data from European XFEL
https://karabo-data.readthedocs.io/
BSD 3-Clause "New" or "Revised" License
13 stars 7 forks source link

'Virtual' stack to hold detector modules #196

Closed takluyver closed 5 years ago

takluyver commented 5 years ago

To avoid an extra copy of detector data which is going to be assembled. The VirtualStack class is like an array with very limited functionality - but enough to pass into the image assembly code. This is meant to address #188.

Other name suggestions are welcome - 'virtual' could get confusing with HDF5 virtual datasets.

I tested this with some a train of 176 pulses from AGIPD. Stacking into a real array took ~360 ms, and assembling that took ~860 ms. Using a virtual array, stacking took << 1 ms, and assembling was roughly unchanged. So in this case it's about a 30% gain, excluding any time to load the data.

tmichela commented 5 years ago

StackView?

zhujun98 commented 5 years ago

LGTM! Thanks a lot for implementing this.

takluyver commented 5 years ago

No problem. Any thoughts on the naming? I'm coming round to something like StackView or StackedView, as @tmichela suggested.

takluyver commented 5 years ago

The class is now called StackView, and the parameter for stack_detector_data() is real_array=False. I'm not entirely happy with that parameter name: anyone have a better idea?

I considered copy=False, but I don't want to give the impression that you can assemble a real numpy array as a view of several disjoint chunks of data.

zhujun98 commented 5 years ago

Maybe view=True?

takluyver commented 5 years ago

I'm not so keen on view=True, because Numpy already uses the term 'view' quite a lot, and it's not a numpy view we're creating.

tmichela commented 5 years ago

I am fine with real_array, unless you found something you like better... LGTM

takluyver commented 5 years ago

I haven't thought of anything better. I'll merge it with real_array as the parameter name. We can still change it if we have a better idea later.