search

quixdb / squash

Compression abstraction library and utilities
https://quixdb.github.io/squash/
MIT License
405 stars 53 forks source link

Lzturbo is missing #205

Closed c324988 closed 8 years ago

c324988 commented 8 years ago

https://sites.google.com/site/powturbo/downloads - binaries https://sites.google.com/site/powturbo/home/benchmark - info

nemequ commented 8 years ago

If someone wants to write a plugin for it, i'd be happy to distribute it in the squash-plugins-extra repository. I don't want to include proprietary software in the main repository, but that's one of the main reasons I created squash-plugins-extra (it's basically for libraries which are either proprietary, unportable, or unmaintained with known security vulnerabilities).

AFAICT, lzturbo is only distributed an executable. It would be possible to create a plugin which just executes lzturbo as a separate process, but this will incur a significant performance penalty if you're trying to compress lots of small files (just like running lzturbo many times from the command line would, there isn't really much we can do), especially on Windows. That means the only one in a position to make a good plugin right now is Hamid… assuming lzturbo is thread-safe, he could just roll everything into the module (so/dll/dynlib).

Also, keep in mind that there is also an issue with portability; right now it's x86_64-only, so it's not possible to run on x86, ARM, POWER, MIPS, etc.

nemequ commented 8 years ago

I don't want it to sound like I'm picking on lzturbo; there are a few other interesting proprietary libraries in similar situations… Oodle, IPP, LZO Professional, etc. There is nothing technical preventing the developers of those libraries (or a third party) from creating or distributing a Squash plugin, especially once Squash is API/ABI stable. If the Squash plugin is open-source but it uses a proprietary library, I'm even willing to host it in squash-plugins-extra and help with maintenance.