prjemian / punx

Python Utilities for NeXus HDF5 files
https://prjemian.github.io/punx
5 stars 7 forks source link

IndexError reading suitcase file #112

Closed prjemian closed 6 years ago

prjemian commented 6 years ago

When trying to get tree structure from an HDF5 prepared by a working version of the NeXus file writer for Bluesky, this error occurs (v0.2.1 of punx):

(base) jemian@otz ~/Documents $ punx tr gov_5.h5

!!!
!!!WARNING: this program is not ready for distribution.
!!!

Traceback (most recent call last):
  File "/home/oxygen/JEMIAN/Apps/anaconda/bin/punx", line 11, in <module>
    sys.exit(main())
  File "/local/Apps/anaconda/lib/python2.7/site-packages/punx/main.py", line 456, in main
    args.func(args)
  File "/local/Apps/anaconda/lib/python2.7/site-packages/punx/main.py", line 199, in func_tree
    report = mc.report(args.show_attributes)
  File "/local/Apps/anaconda/lib/python2.7/site-packages/punx/h5tree.py", line 66, in report
    tree_string_list = self._renderGroup(f, txt, indentation = "")
  File "/local/Apps/anaconda/lib/python2.7/site-packages/punx/h5tree.py", line 117, in _renderGroup
    s += self._renderGroup(value, itemname, indentation+"  ")
  File "/local/Apps/anaconda/lib/python2.7/site-packages/punx/h5tree.py", line 117, in _renderGroup
    s += self._renderGroup(value, itemname, indentation+"  ")
  File "/local/Apps/anaconda/lib/python2.7/site-packages/punx/h5tree.py", line 105, in _renderGroup
    s += self._renderDataset(value, itemname, indentation+"  ")
  File "/local/Apps/anaconda/lib/python2.7/site-packages/punx/h5tree.py", line 162, in _renderDataset
    s += self._renderAttributes(dset, indentation)
  File "/local/Apps/anaconda/lib/python2.7/site-packages/punx/h5tree.py", line 126, in _renderAttributes
    s.append("%s  @%s = %s" % (indentation, name, utils.decode_byte_string(value)))
  File "/local/Apps/anaconda/lib/python2.7/site-packages/punx/utils.py", line 44, in decode_byte_string
    text = text[0]
IndexError: index 0 is out of bounds for axis 0 with size 0

test file gov_5.h5 will be deposited in the repo

The file passes validation:

(base) jemian@otz ~/Documents $ punx va gov_5.h5 

!!!
!!!WARNING: this program is not ready for distribution.
!!!

data file: gov_5.h5
NeXus definitions (release): v2018.5, dated 2018-05-15 16:34:19, sha=a3045fd7ff0029adc811d9ed92f42ad51007f32d

findings
=================================================== ====== ==================================== ========================================================================================
address                                             status test                                 comments                                                                                
=================================================== ====== ==================================== ========================================================================================
/                                                   TODO   NeXus base class                     NXroot: more validations needed                                                         
/                                                   OK     known NXDL                           NXroot: recognized NXDL specification                                                   
/                                                   OK     NeXus base class                     NXroot: known NeXus base class                                                          
/                                                   OK     NeXus default plot                   found by v3: /gov_5/primary_data@signal                                                 
/@HDF5_Version                                      TODO   attribute value                      implement                                                                               
/@HDF5_Version                                      TODO   value of @HDF5_Version               TODO: need to validate: @HDF5_Version = 1.8.17                                          
/@HDF5_Version                                      OK     validItemName                        relaxed pattern: [A-Za-z_][\w_]*                                                        
/@HDF5_Version                                      OK     known attribute                      known: NXroot@HDF5_Version                                                              
/@creator                                           TODO   attribute value                      implement                                                                               
/@creator                                           TODO   value of @creator                    TODO: need to validate: @creator = https://github.com/NSLS-II/suitcase/suitcase/nexus.py
/@creator                                           OK     validItemName                        strict pattern: [a-z_][a-z0-9_]*                                                        
/@creator                                           OK     known attribute                      known: NXroot@creator                                                                   
/@default                                           TODO   attribute value                      implement                                                                               
/@default                                           OK     validItemName                        strict pattern: [a-z_][a-z0-9_]*                                                        
/@default                                           OK     known attribute                      known: NXroot@default                                                                   
/@default                                           OK     value of @default                    found: /gov_5                                                                           
/@file_name                                         TODO   attribute value                      implement                                                                               
/@file_name                                         TODO   value of @file_name                  TODO: need to validate: @file_name = gov_5.h5                                           
/@file_name                                         OK     validItemName                        strict pattern: [a-z_][a-z0-9_]*                                                        
/@file_name                                         OK     known attribute                      known: NXroot@file_name                                                                 
/@file_time                                         TODO   attribute value                      implement                                                                               
/@file_time                                         TODO   value of @file_time                  TODO: need to validate: @file_time = 2017-03-28 10:16:54.123762                         
/@file_time                                         OK     validItemName                        strict pattern: [a-z_][a-z0-9_]*                                                        
/@file_time                                         OK     known attribute                      known: NXroot@file_time                                                                 
/gov_5                                              TODO   NeXus base class                     NXentry: more validations needed                                                        
/gov_5                                              OK     validItemName                        strict pattern: [a-z_][a-z0-9_]*                                                        
/gov_5                                              OK     group in base class                  not defined: NXroot/gov_5                                                               
/gov_5                                              OK     known NXDL                           NXentry: recognized NXDL specification                                                  
/gov_5                                              OK     NeXus base class                     NXentry: known NeXus base class                                                         
/gov_5@NX_class                                     OK     validItemName                        pattern: NX.+                                                                           
/gov_5@NX_class                                     OK     attribute value                      recognized NXDL base class: NXentry                                                     
/gov_5@NX_class                                     OK     known attribute                      known: NXentry@NX_class                                                                 
/gov_5@_BlueSky__name                               TODO   attribute value                      implement                                                                               
/gov_5@_BlueSky__name                               OK     validItemName                        relaxed pattern: [A-Za-z_][\w_]*                                                        
/gov_5@_BlueSky__name                               OK     known attribute                      unknown: NXentry@_BlueSky__name                                                         
/gov_5@_BlueSky_start                               TODO   attribute value                      implement                                                                               
/gov_5@_BlueSky_start                               OK     validItemName                        relaxed pattern: [A-Za-z_][\w_]*                                                        
/gov_5@_BlueSky_start                               OK     known attribute                      unknown: NXentry@_BlueSky_start                                                         
/gov_5@_BlueSky_stop                                TODO   attribute value                      implement                                                                               
/gov_5@_BlueSky_stop                                OK     validItemName                        relaxed pattern: [A-Za-z_][\w_]*                                                        
/gov_5@_BlueSky_stop                                OK     known attribute                      unknown: NXentry@_BlueSky_stop                                                          
/gov_5@default                                      TODO   attribute value                      implement                                                                               
/gov_5@default                                      OK     validItemName                        strict pattern: [a-z_][a-z0-9_]*                                                        
/gov_5@default                                      OK     known attribute                      known: NXentry@default                                                                  
/gov_5@default                                      OK     value of @default                    found: /gov_5/primary_data                                                              
/gov_5/primary                                      TODO   NeXus base class                     NXlog: more validations needed                                                          
/gov_5/primary                                      OK     validItemName                        strict pattern: [a-z_][a-z0-9_]*                                                        
/gov_5/primary                                      OK     group in base class                  not defined: NXentry/primary                                                            
/gov_5/primary                                      OK     known NXDL                           NXlog: recognized NXDL specification                                                    
/gov_5/primary                                      OK     NeXus base class                     NXlog: known NeXus base class                                                           
/gov_5/primary                                      OK     NXDL field in data file              found: /gov_5/primary/time                                                              
/gov_5/primary@NX_class                             OK     validItemName                        pattern: NX.+                                                                           
/gov_5/primary@NX_class                             OK     attribute value                      recognized NXDL base class: NXlog                                                       
/gov_5/primary@NX_class                             OK     known attribute                      known: NXlog@NX_class                                                                   
/gov_5/primary@_BlueSky_configuration               TODO   attribute value                      implement                                                                               
/gov_5/primary@_BlueSky_configuration               OK     validItemName                        relaxed pattern: [A-Za-z_][\w_]*                                                        
/gov_5/primary@_BlueSky_configuration               OK     known attribute                      unknown: NXlog@_BlueSky_configuration                                                   
/gov_5/primary@_BlueSky_name                        TODO   attribute value                      implement                                                                               
/gov_5/primary@_BlueSky_name                        OK     validItemName                        relaxed pattern: [A-Za-z_][\w_]*                                                        
/gov_5/primary@_BlueSky_name                        OK     known attribute                      unknown: NXlog@_BlueSky_name                                                            
/gov_5/primary@_BlueSky_object_keys                 TODO   attribute value                      implement                                                                               
/gov_5/primary@_BlueSky_object_keys                 OK     validItemName                        relaxed pattern: [A-Za-z_][\w_]*                                                        
/gov_5/primary@_BlueSky_object_keys                 OK     known attribute                      unknown: NXlog@_BlueSky_object_keys                                                     
/gov_5/primary@_BlueSky_run_start                   TODO   attribute value                      implement                                                                               
/gov_5/primary@_BlueSky_run_start                   OK     validItemName                        relaxed pattern: [A-Za-z_][\w_]*                                                        
/gov_5/primary@_BlueSky_run_start                   OK     known attribute                      unknown: NXlog@_BlueSky_run_start                                                       
/gov_5/primary@_BlueSky_time                        TODO   attribute value                      implement                                                                               
/gov_5/primary@_BlueSky_time                        OK     validItemName                        relaxed pattern: [A-Za-z_][\w_]*                                                        
/gov_5/primary@_BlueSky_time                        OK     known attribute                      unknown: NXlog@_BlueSky_time                                                            
/gov_5/primary@_BlueSky_uid                         TODO   attribute value                      implement                                                                               
/gov_5/primary@_BlueSky_uid                         OK     validItemName                        relaxed pattern: [A-Za-z_][\w_]*                                                        
/gov_5/primary@_BlueSky_uid                         OK     known attribute                      unknown: NXlog@_BlueSky_uid                                                             
/gov_5/primary/noisy                                OK     validItemName                        strict pattern: [a-z_][a-z0-9_]*                                                        
/gov_5/primary/noisy                                OK     field in base class                  not defined: NXlog/noisy                                                                
/gov_5/primary/noisy@_BlueSky_dtype                 TODO   attribute value                      implement                                                                               
/gov_5/primary/noisy@_BlueSky_dtype                 OK     validItemName                        relaxed pattern: [A-Za-z_][\w_]*                                                        
/gov_5/primary/noisy@_BlueSky_lower_ctrl_limit      TODO   attribute value                      implement                                                                               
/gov_5/primary/noisy@_BlueSky_lower_ctrl_limit      OK     validItemName                        relaxed pattern: [A-Za-z_][\w_]*                                                        
/gov_5/primary/noisy@_BlueSky_object_name           TODO   attribute value                      implement                                                                               
/gov_5/primary/noisy@_BlueSky_object_name           OK     validItemName                        relaxed pattern: [A-Za-z_][\w_]*                                                        
/gov_5/primary/noisy@_BlueSky_precision             TODO   attribute value                      implement                                                                               
/gov_5/primary/noisy@_BlueSky_precision             OK     validItemName                        relaxed pattern: [A-Za-z_][\w_]*                                                        
/gov_5/primary/noisy@_BlueSky_shape                 TODO   attribute value                      implement                                                                               
/gov_5/primary/noisy@_BlueSky_shape                 OK     validItemName                        relaxed pattern: [A-Za-z_][\w_]*                                                        
/gov_5/primary/noisy@_BlueSky_source                TODO   attribute value                      implement                                                                               
/gov_5/primary/noisy@_BlueSky_source                OK     validItemName                        relaxed pattern: [A-Za-z_][\w_]*                                                        
/gov_5/primary/noisy@_BlueSky_units                 TODO   attribute value                      implement                                                                               
/gov_5/primary/noisy@_BlueSky_units                 OK     validItemName                        relaxed pattern: [A-Za-z_][\w_]*                                                        
/gov_5/primary/noisy@_BlueSky_upper_ctrl_limit      TODO   attribute value                      implement                                                                               
/gov_5/primary/noisy@_BlueSky_upper_ctrl_limit      OK     validItemName                        relaxed pattern: [A-Za-z_][\w_]*                                                        
/gov_5/primary/noisy@axes                           TODO   attribute value                      implement                                                                               
/gov_5/primary/noisy@axes                           OK     validItemName                        strict pattern: [a-z_][a-z0-9_]*                                                        
/gov_5/primary/noisy@key_name                       TODO   attribute value                      implement                                                                               
/gov_5/primary/noisy@key_name                       OK     validItemName                        strict pattern: [a-z_][a-z0-9_]*                                                        
/gov_5/primary/noisy@target                         OK     validItemName                        strict pattern: [a-z_][a-z0-9_]*                                                        
/gov_5/primary/noisy@target                         OK     attribute value                      found: @target=/gov_5/primary/noisy                                                     
/gov_5/primary/noisy_timestamps                     OK     validItemName                        strict pattern: [a-z_][a-z0-9_]*                                                        
/gov_5/primary/noisy_timestamps                     OK     field in base class                  not defined: NXlog/noisy_timestamps                                                     
/gov_5/primary/noisy_timestamps@key_name            TODO   attribute value                      implement                                                                               
/gov_5/primary/noisy_timestamps@key_name            OK     validItemName                        strict pattern: [a-z_][a-z0-9_]*                                                        
/gov_5/primary/noisy_timestamps@target              OK     validItemName                        strict pattern: [a-z_][a-z0-9_]*                                                        
/gov_5/primary/noisy_timestamps@target              OK     attribute value                      found: @target=/gov_5/primary/noisy_timestamps                                          
/gov_5/primary/time                                 OK     validItemName                        strict pattern: [a-z_][a-z0-9_]*                                                        
/gov_5/primary/time                                 OK     field in base class                  defined: NXlog/time                                                                     
/gov_5/primary/time@start                           TODO   attribute value                      implement                                                                               
/gov_5/primary/time@start                           OK     validItemName                        strict pattern: [a-z_][a-z0-9_]*                                                        
/gov_5/primary/time@units                           TODO   attribute value                      implement                                                                               
/gov_5/primary/time@units                           OK     validItemName                        strict pattern: [a-z_][a-z0-9_]*                                                        
/gov_5/primary_data                                 TODO   NeXus base class                     NXdata: more validations needed                                                         
/gov_5/primary_data                                 OK     validItemName                        strict pattern: [a-z_][a-z0-9_]*                                                        
/gov_5/primary_data                                 OK     group in base class                  not defined: NXentry/primary_data                                                       
/gov_5/primary_data                                 OK     known NXDL                           NXdata: recognized NXDL specification                                                   
/gov_5/primary_data                                 OK     NeXus base class                     NXdata: known NeXus base class                                                          
/gov_5/primary_data@NX_class                        OK     validItemName                        pattern: NX.+                                                                           
/gov_5/primary_data@NX_class                        OK     attribute value                      recognized NXDL base class: NXdata                                                      
/gov_5/primary_data@NX_class                        OK     known attribute                      known: NXdata@NX_class                                                                  
/gov_5/primary_data@signal                          OK     validItemName                        strict pattern: [a-z_][a-z0-9_]*                                                        
/gov_5/primary_data@signal                          OK     valid name @signal=noisy             strict pattern: [a-z_][a-z0-9_]*                                                        
/gov_5/primary_data@signal                          OK     attribute value                      found: @signal=noisy                                                                    
/gov_5/primary_data@signal                          OK     known attribute                      known: NXdata@signal                                                                    
/gov_5/primary_data@signal                          OK     value of @signal                     found: /gov_5/primary_data/noisy                                                        
/gov_5/primary_data@signal                          OK     NeXus default plot v3, NXdata@signal correct default plot setup in /NXentry/NXdata                                           
/gov_5/primary_data@signal                          OK     NeXus default plot v3 NIAC2014       default plot setup in /NXentry/NXdata                                                   
/gov_5/primary_data/noisy                           OK     validItemName                        strict pattern: [a-z_][a-z0-9_]*                                                        
/gov_5/primary_data/noisy                           OK     field in base class                  not defined: NXdata/noisy                                                               
/gov_5/primary_data/noisy@_BlueSky_dtype            TODO   attribute value                      implement                                                                               
/gov_5/primary_data/noisy@_BlueSky_lower_ctrl_limit TODO   attribute value                      implement                                                                               
/gov_5/primary_data/noisy@_BlueSky_object_name      TODO   attribute value                      implement                                                                               
/gov_5/primary_data/noisy@_BlueSky_precision        TODO   attribute value                      implement                                                                               
/gov_5/primary_data/noisy@_BlueSky_shape            TODO   attribute value                      implement                                                                               
/gov_5/primary_data/noisy@_BlueSky_source           TODO   attribute value                      implement                                                                               
/gov_5/primary_data/noisy@_BlueSky_units            TODO   attribute value                      implement                                                                               
/gov_5/primary_data/noisy@_BlueSky_upper_ctrl_limit TODO   attribute value                      implement                                                                               
/gov_5/primary_data/noisy@axes                      TODO   attribute value                      implement                                                                               
/gov_5/primary_data/noisy@key_name                  TODO   attribute value                      implement                                                                               
/gov_5/primary_data/noisy@target                    OK     attribute value                      found: @target=/gov_5/primary/noisy                                                     
/gov_5/primary_data/noisy_timestamps                OK     validItemName                        strict pattern: [a-z_][a-z0-9_]*                                                        
/gov_5/primary_data/noisy_timestamps                OK     field in base class                  not defined: NXdata/noisy_timestamps                                                    
/gov_5/primary_data/noisy_timestamps@key_name       TODO   attribute value                      implement                                                                               
/gov_5/primary_data/noisy_timestamps@target         OK     attribute value                      found: @target=/gov_5/primary/noisy_timestamps                                          
=================================================== ====== ==================================== ========================================================================================

summary statistics
======== ===== =========================================================== =========
status   count description                                                 (value)  
======== ===== =========================================================== =========
OK       93    meets NeXus specification                                   100      
NOTE     0     does not meet NeXus specification, but acceptable           75       
WARN     0     does not meet NeXus specification, not generally acceptable 25       
ERROR    0     violates NeXus specification                                -10000000
TODO     47    validation not implemented yet                              0        
UNUSED   0     optional NeXus item not used in data file                   0        
COMMENT  0     comment from the punx source code                           0        
OPTIONAL 49    allowed by NeXus specification, not identified              99       
         --                                                                         
TOTAL    189                                                                        
======== ===== =========================================================== =========

<value>/finding=99.654930  count=142  sum(finding values)=14151.000000