appier / h5sparse

Scipy sparse matrix in HDF5
https://pypi.python.org/pypi/h5sparse
MIT License
45 stars 8 forks source link

Can't add other data into sparse #7

Closed duoan closed 6 years ago

duoan commented 6 years ago

Thanks @appier I can add sparse data with h5sparse, but can add array or list and other type data into h5sparse File.

import h5sparse
def dump_h5py(dump_file, matrix, fnames):
    with h5sparse.File(dump_file) as h5f:
        h5f.create_dataset('matrix', data=matrix)
        h5f.create_dataset('fnames', data=fnames)
duoan commented 6 years ago
~/.pyenv/versions/stats36/lib/python3.6/site-packages/h5sparse/h5sparse.py in create_dataset(self, name, shape, dtype, data, format, indptr_dtype, indices_dtype, **kwargs)
     69         else:
     70             group = self.h5py_group.create_group(name)
---> 71             group.attrs['h5sparse_format'] = get_format_str(data)
     72             group.attrs['h5sparse_shape'] = data.shape
     73             group.create_dataset('data', data=data.data, dtype=dtype, **kwargs)

~/.pyenv/versions/stats36/lib/python3.6/site-packages/h5sparse/h5sparse.py in get_format_str(data)
     15         if isinstance(data, format_class):
     16             return format_str
---> 17     raise ValueError("Data type {} is not supported.".format(type(data)))
     18 
     19 

ValueError: Data type <class 'list'> is not supported.
duoan commented 6 years ago

I mean h5sparse is a extend for h5py added sparse data feature and support other all feature like h5py.

If I'm wrong, please correct me.

ianlini commented 6 years ago

We currently only support Scipy sparse matrix. It's simply a POC about combining HDF5 with sparse matrix. We still need much more effort to integrate with full h5py functions, but we are not actively contributing to this project now.

ianlini commented 6 years ago

implemented in #8