search

prjemian / punx

Python Utilities for NeXus HDF5 files
https://prjemian.github.io/punx
5 stars 7 forks source link

Error when validating proposed NX data file #124

Closed prjemian closed 4 years ago

prjemian commented 4 years ago

When attempting to validate a proposed NeXus HDF5 data file as an example of a specific application definition, this KeyError exception was raised:

(base) mintadmin@mint-vm:~/Documents/Desktop$ punx valid NXstxm.hdf5

!!! WARNING: this program is not ready for distribution.

/home/mintadmin/Documents/eclipse/punx/src/punx/validations/application_definition.py:21: H5pyDeprecationWarning: dataset.value has been deprecated. Use dataset[()] instead.
  ad_name = utils.decode_byte_string(v_item.h5_object["definition"].value)
/home/mintadmin/Documents/eclipse/punx/src/punx/validations/application_definition.py:60: H5pyDeprecationWarning: dataset.value has been deprecated. Use dataset[()] instead.
  found = enum == utils.decode_byte_string(h5_obj.value)
Traceback (most recent call last):
  File "/home/mintadmin/Apps/anaconda/bin/punx", line 11, in <module>
    load_entry_point('punx', 'console_scripts', 'punx')()
  File "/home/mintadmin/Documents/eclipse/punx/src/punx/main.py", line 456, in main
    args.func(args)
  File "/home/mintadmin/Documents/eclipse/punx/src/punx/main.py", line 236, in func_validate
    validator.validate(args.infile)
  File "/home/mintadmin/Documents/eclipse/punx/src/punx/validate.py", line 256, in validate
    default_plot.verify(self)
  File "/home/mintadmin/Documents/eclipse/punx/src/punx/validations/default_plot.py", line 33, in verify
    addr = method(validator)
  File "/home/mintadmin/Documents/eclipse/punx/src/punx/validations/default_plot.py", line 101, in default_plot_v3
    t4 = utils.isNeXusDataset(validator.addresses[signal_h5_addr].h5_object)
KeyError: '/ENTRY/DATA/<signal>_name_here must match one of the field names in this group'

Here is the tree structure:

``` (base) mintadmin@mint-vm:~/Documents/Desktop$ punx tree NXstxm.hdf5 !!! WARNING: this program is not ready for distribution. /home/mintadmin/Documents/eclipse/punx/src/punx/h5tree.py:155: H5pyDeprecationWarning: dataset.value has been deprecated. Use dataset[()] instead. value = " = %s" % str(dset.value) /media/sf_C_DRIVE/Users/Pete/Desktop/NXstxm.hdf5 : NeXus data file @doc = Application definition for a STXM instrument. The interferometer position measurements, monochromator photon energy values and detector measurements are all treated as NXdetectors and stored within the NXinstrument group as lists of values stored in chronological order. The NXdata group then holds another version of the data in a regular 3D array (NumE by NumY by NumX, for a total of NumP points in a sample image stack type scan). The former data values should be stored with a minimum loss of precision, while the latter values can be simplified and/or approximated in order to fit the constraints of a regular 3D array. 'Line scans' and 'point spectra' are just sample_image scan types with reduced dimensions in the same way as single images have reduced E dimensions compared to image 'stacks'. @file_name = /home/bergr/github/nexusformat/exampledata/nxdl/applications/NXstxm.hdf5 @file_time = 2020-02-27T16:01:29.861604 @numE = number of photon energies scanned @numP = total number of scan points @numX = number of pixels in X direction @numY = number of pixels in Y direction ENTRY:NXentry @NX_class = NXentry @required = True definition:NX_CHAR = NXstxm @NX_class = NX_CHAR @NX_units = NX_ANY @doc = Official NeXus NXDL schema to which this file conforms @enumerations = NXstxm @required = True end_time:NX_CHAR = 2020-02-27T16:01:29.857356 @NX_class = NX_DATE_TIME @NX_units = NX_ANY @required = True start_time:NX_CHAR = 2020-02-27T16:01:29.857345 @NX_class = NX_DATE_TIME @NX_units = NX_ANY @required = True title:NX_CHAR = @NX_class = NX_CHAR @NX_units = NX_ANY @required = True DATA:NXdata @NX_class = NXdata @axes = _name_here must match one of the field names in this group @required = True @signal = _name_here must match one of the field names in this group data:NX_FLOAT64[] = @NX_class = NX_NUMBER @NX_units = NX_ANY @doc = Detectors that provide more than one value per scan point should be summarised to a single value per scan point for this array in order to simplify plotting. Note that 'Line scans' and focus type scans measure along one spatial dimension but are not restricted to being parallel to the X or Y axes. Such scans should therefore use a single dimension for the positions along the spatial line. The 'sample_x' and 'sample_y' fields should then contain lists of the x- and y-positions and should both have the 'axis' attribute pointing to the same dimension. @required = True energy:NX_FLOAT64 = 1.0 @NX_class = NX_FLOAT @NX_units = NX_ANY @doc = List of photon energies of the X-ray beam. If scanned through multiple values, then an 'axis' attribute will be required to link the field to the appropriate data array dimension. @required = True sample_x:NX_FLOAT64 = 1.0 @NX_class = NX_FLOAT @NX_units = NX_ANY @doc = List of X positions on the sample. If scanned through multiple values, then an 'axis' attribute will be required to link the field to the appropriate data array dimension. @required = True sample_y:NX_FLOAT64 = 1.0 @NX_class = NX_FLOAT @NX_units = NX_ANY @doc = List of Y positions on the sample. If scanned through multiple values, then an 'axis' attribute will be required to link the field to the appropriate data array dimension. @required = True stxm_scan_type:NX_CHAR = sample point spectrum @NX_class = NX_CHAR @NX_units = NX_ANY @doc = Label for typical scan types as a convenience for humans. Each label corresponds to a specific set of axes being scanned to produce a data array of shape: * sample point spectrum: (photon_energy,) * sample line spectrum: (photon_energy, sample_y/sample_x) * sample image: (sample_y, sample_x) * sample image stack: (photon_energy, sample_y, sample_x) * sample focus: (zoneplate_z, sample_y/sample_x) * osa image: (osa_y, osa_x) * osa focus: (zoneplate_z, osa_y/osa_x) * detector image: (detector_y, detector_x) The "generic scan" string is to be used when none of the other choices are appropriate. @enumerations = sample point spectrum @required = True INSTRUMENT:NXinstrument @NX_class = NXinstrument @required = True DETECTOR:NXdetector @NX_class = NXdetector @required = True data:NX_FLOAT64 = 1.0 @NX_class = NX_NUMBER @NX_units = NX_ANY @required = True SOURCE:NXsource @NX_class = NXsource @required = True name:NX_CHAR = @NX_class = NX_CHAR @NX_units = NX_ANY @required = True probe:NX_CHAR = @NX_class = NX_CHAR @NX_units = NX_ANY @required = True type:NX_CHAR = @NX_class = NX_CHAR @NX_units = NX_ANY @required = True monochromator:NXmonochromator @NX_class = NXmonochromator @required = True energy:NX_FLOAT64 = 1.0 @NX_class = NX_FLOAT @NX_units = NX_ANY @required = True sample_x:NXdetector @NX_class = NXdetector @doc = Measurements of the sample position from the x-axis interferometer. @required = False data:NX_FLOAT64 = 1.0 @NX_class = NX_FLOAT @NX_units = NX_ANY @required = True sample_y:NXdetector @NX_class = NXdetector @doc = Measurements of the sample position from the y-axis interferometer. @required = False data:NX_FLOAT64 = 1.0 @NX_class = NX_FLOAT @NX_units = NX_ANY @required = True sample_z:NXdetector @NX_class = NXdetector @doc = Measurements of the sample position from the z-axis interferometer. @required = False data:NX_FLOAT64 = 1.0 @NX_class = NX_FLOAT @NX_units = NX_ANY @required = True SAMPLE:NXsample @NX_class = NXsample @required = True rotation_angle:NX_FLOAT64[] = @NX_class = NX_FLOAT @NX_units = NX_ANY @required = True control:NXmonitor @NX_class = NXmonitor @required = False data:NX_FLOAT64[] = @NX_class = NX_FLOAT @NX_units = NX_ANY @doc = Values to use to normalise for time-variations in photon flux. Typically, the synchrotron storage ring electron beam current is used as a proxy for the X-ray beam intensity. Array must have same shape as the NXdata groups. @required = True ```
prjemian commented 4 years ago

Aha! Found the problem while replying to author's private email. This is the problem:

/ENTRY/DATA@signal= "<signal>_name_here must match one of the field names in this group"

yet there is no such field in the /ENTRY/DATA group named <signal>_name_here must match one of the field names in this group so this attribute is the cause of the validation error.

The error report from punx is precise but hard for the user to understand since the expected field name looks like program documentation.

prjemian commented 4 years ago

Could rewrite that KeyError message to better indicate exact group name and exact field name as requested.