search

sergey-dryabzhinsky / python-zstd

Simple python bindings to Yann Collet ZSTD compression library
BSD 2-Clause "Simplified" License
165 stars 27 forks source link

General code cleanups prior to adding streaming compression #31

Closed zackw closed 6 years ago

zackw commented 6 years ago

I plan to add streaming compression and decompression with an API similar to what the Python standard library has for gzip, bzip2, and lzma (e.g. lzma.LZMACompressor). But before tackling something big like that, to familiarize myself with the existing code, I went through and made a whole lot of small cleanups and improvements. Docstrings for everything, better type safety and less ifdeffage in the C component, define as much as possible in Python rather than C, deprecation warnings, more thorough testing, that sort of thing. Please let me know what you think.

sergey-dryabzhinsky commented 6 years ago
  1. Too much changes in one pull.
  2. I don't plan to add anything to current api. No streaming, no dicts. You should use https://github.com/indygreg/python-zstandard for that.
zackw commented 6 years ago

I'd be willing to split up the pull request if it will make it easier to review.

However, I'm only going to do the work of that if you're willing to reconsider adding basic streaming support. Right now python-zstd cannot decompress data compressed by piping to the zstd command line tool, and it can't provide the equivalent of gzip.open either, and both of those are essential for what I want to use it for. python-zstandard is much more complicated than I need, and also buggy, and the complexity makes it too hard for me to fix the bugs.