kiyo-masui / bitshuffle

Filter for improving compression of typed binary data.
Other
219 stars 76 forks source link

dtype argument issue #75

Closed hhslepicka closed 5 years ago

hhslepicka commented 5 years ago

I'm trying to decompress a numpy array and if I send in the dtype from the array itself I get a TypeError: an integer is required. If I send dtype as an integer representing the size in bytes needed by my original data type, it fails TypeError: data type not understood.

So far I was able to fix it by creating a wrapper around the dtype like so:

class DTypeWrapper(object):
    def __init__(self, dtype):
        self.dtype = dtype

    def __getattr__(self, item):
        if item == 'itemsize':
            return np.zeros(1, dtype=self.dtype).itemsize
        return self.dtype.__getattr__(item)

def bslz4_decompress(data, shape, dtype):
    nelems = reduce((lambda x, y: x * y), shape)
    dec_data = bitshuffle.decompress_lz4(data, (nelems,), DTypeWrapper(dtype))
    return dec_data.reshape(shape)
hhslepicka commented 5 years ago

Found out that I was sending in the wrong stuff. I should use the instance of the dtype and not the dtype itself. Which means:

    dec_data = bitshuffle.decompress_lz4(data, (nelems,), np.dtype(dtype))