search

zarr-developers / numcodecs

A Python package providing buffer compression and transformation codecs for use in data storage and communication applications.
http://numcodecs.readthedocs.io
MIT License
125 stars 88 forks source link

Refactor decode implementation in vlen codecs #135

Open alimanfoo opened 5 years ago

alimanfoo commented 5 years ago

As noticed in #81, there is a lot of code duplication in the implementation of the decode() methods in the vlen codecs. Would be good to refactor and remove code duplication.

jeromekelleher commented 5 years ago

I had a quick crack at this, and it mostly works (we just need a def _decode_item(self, data, size) method defined in sub classes). However, there's some Cython trickiness involved and this didn't work as expected for the Array version. I don't know enough about Cython to dig into it I'm afraid, and I really must stop procrastinating from what I'm actually supposed to be doing!

alimanfoo commented 5 years ago

I had a quick crack at this, and it mostly works (we just need a def _decode_item(self, data, size) method defined in sub classes). However, there's some Cython trickiness involved and this didn't work as expected for the Array version. I don't know enough about Cython to dig into it I'm afraid, and I really must stop procrastinating from what I'm actually supposed to be doing!

Yeah, don't you have a proposal deadline :smiley:

Would be cool if you could whack whatever you have in a PR, just to take a look.

jeromekelleher commented 5 years ago

Ack, I nuked it all. It's very simple though, just adding a VlenBase class, putting the decode method in there and then having an implementation of def _decode_item(self, data, size) in each subclass. Worked fine for the non-array subclasses.

alimanfoo commented 5 years ago

:smile: no worries, thanks for the info.