Can't read HDF5 created with pandas.

[joneugster]

Hi,

Your tool looks amazing! Unfortunately it seems not to like my HDF5 files, created with pandas.

Maybe you could implement that, too. Best regards!

Reproduction:

Execute this in Python to create the HDF5 file:

import pandas as pd
import numpy as np
with pd.HDFStore('NOT_IMPLEMENTED.h5') as store:
    df = pd.DataFrame(np.random.rand(4, 4),
                           columns=['A', 'B', 'C', 'D'],
                           index=pd.date_range("20180101 00:00", periods=4))
    store.put('Test', df, format='table', data_columns=True)

Then open it in argos (argos NOT_IMPLEMENTED.h5) and open the group "Test" in the treeview.

Error Message

Bug: uncaught OSError Unable to read attribute (no appropriate function for conversion path)

``` Traceback (most recent call last): File "/anaconda3/lib/python3.7/site-packages/argos/qt/treemodels.py", line 91, in data return self.itemData(item, index.column(), role=role) File "/anaconda3/lib/python3.7/site-packages/argos/repo/repotreemodel.py", line 113, in itemData return super(RepoTreeModel, self).itemData(treeItem, column, role=role) File "/anaconda3/lib/python3.7/site-packages/argos/qt/treemodels.py", line 117, in itemData return item.decoration File "/anaconda3/lib/python3.7/site-packages/argos/repo/baserti.py", line 274, in decoration return rtiIconFactory.getIcon(self.iconGlyph, isOpen=not self.canFetchChildren(), File "/anaconda3/lib/python3.7/site-packages/argos/repo/rtiplugins/hdf5.py", line 359, in iconGlyph if self._h5Dataset.attrs.get('CLASS', None) == b'DIMENSION_SCALE': File "/anaconda3/lib/python3.7/_collections_abc.py", line 660, in get return self[key] File "h5py/_objects.pyx", line 54, in h5py._objects.with_phil.wrapper File "h5py/_objects.pyx", line 55, in h5py._objects.with_phil.wrapper File "/anaconda3/lib/python3.7/site-packages/h5py/_hl/attrs.py", line 81, in __getitem__ attr.read(arr, mtype=htype) File "h5py/_objects.pyx", line 54, in h5py._objects.with_phil.wrapper File "h5py/_objects.pyx", line 55, in h5py._objects.with_phil.wrapper File "h5py/h5a.pyx", line 355, in h5py.h5a.AttrID.read File "h5py/_proxy.pyx", line 36, in h5py._proxy.attr_rw OSError: Unable to read attribute (no appropriate function for conversion path) ```

[titusjan]

Technical explanation: the problem seems to be that PyTables (which is used by Pandas to create the HDF files) stores some string attributes as fixed-length utf-8 strings. These can not be read by the H5Py library that Argos uses to read the HDF-5 files.

The best solution would be that the issue is fixed in H5Py. Unfortunately the bug is more that two years old so I don't see it being fixed soon. See https://github.com/h5py/h5py/issues/585

I've made a work-around so that Argos at least open the file, but some attributes cannot be read. See the screenshot below.

The work around is in the development branch. You can test try that if you like.

[joneugster]

Wonderful, thanks!

To be honest, I assume that most users don't need those attributes anyways, it is usually more important to see the values of the data.

Now I only have an import error coming from not finding pgcolorbar, but I can download that from your other repo.