search

bluesky / suitcase-core

data export facilities for NSLS-II
https://blueskyproject.io/suitcase
Other
2 stars 13 forks source link

BUG: mesh scan (outer_product_scan) HDF5 data not saved as 2-D #26

Open prjemian opened 7 years ago

prjemian commented 7 years ago

In suitcase.hdf5.export(), data from a mesh scan has not been saved as 2-D. Here's an example:

outer_product_scan([det], m1, -3, 3, 25, m2, -2, 2, 20, True)

Also, the scanned axes should be available as arrays of m1 with 1-D length of 25 and m2 with 1-D length of 20.

Suggest applying a numpy.reshape technique such as implemented in spec2nexus.utils.reshape_data.

The outer_product_scan() considers the possibility of jittery positioners. But the lists of requested positions for m1 and m2 should be written so as to enable the simplest visualization where the independent axes of the mesh can be scaled by the requested positioner values.

tacaswell commented 7 years ago

This is to some degree a feature, the data is just a stream of events and the current implementation of suitcase directly reflects how the raw-data. The export just turns naively turns these into a collection of columns. In general, we do not know anything about the semantics of the data (ex, which column is the intensity, how to re-compute what the requested axes are, what the shape is, ...) so this is the safe guaranteed-to-work

It seems reasonable to make export more sophisticated to look at the header and do something more sophisticated (and maybe lossy!) to export to a specific file format. We have a skeleton of how to do this for a GUI (https://github.com/NSLS-II/qtbroker ) but it is not immediately obvious to me what the correct factorization for export is. As a first pass export_mesh_to_nexus(h : Header, fname: Path) would be the right place to start (which would blow up if it can not extract enough information from the header to do the rasterization, sort out scales etc).

prjemian commented 7 years ago

agreed, it's more an ENH than BUG with content in the start document, the axes and their ranges can be found thus the expected shape of the data (it might be an incomplete set if aborted early), the "signal" (default plot to be made) is ambiguous (pick the first one is one method, SPEC picks the last one by its convention, user will become accustomed to the pattern)

It is possible. If it fails, deposit the data as series of raw events - the current method. I'll target this for the nexus.py fork+branch and we'll see if it is worth trying to generalize