numcodecs compression (#60)

gallantlab / cottoncandy

sugar for s3

http://gallantlab.github.io/cottoncandy/

BSD 2-Clause "Simplified" License

33 stars 16 forks source link

Closed marklescroart closed 5 years ago

marklescroart commented 5 years ago

Added numcodecs compression options
Backward compatibility for download of raw arrays w/ gzip=True
Modify download_raw_array to be memory-efficient while decompressing numcodecs arrays

All changes from prev PR from forked repo, now here. All seems to work well, no memory overhead.

anwarnunez commented 5 years ago

Very cool, thanks! It looks like there is on missing numcodec import still: https://travis-ci.org/gallantlab/cottoncandy/jobs/403460114

marklescroart commented 5 years ago

Wasn't missing import, was missing numcodecs in requirements.txt. This push should fix it...