Add LZO

[martindurant]

A rarer compression algorithm for parquet is LZO. This library seems to do it: https://badboy.github.io/minilzo-rs/minilzo/index.html

[messense]

Note that there is a pure Rust version: https://github.com/gmg137/minilzo-rs

Never mind, the project's description A pure rust implementation bound to the C version of minilzo. is confusing.

[martindurant]

GPL might be a problem for this project.

On February 28, 2021 6:27:17 AM EST, messense notifications@github.com wrote:

Note that there is a pure Rust version: https://github.com/gmg137/minilzo-rs

-- You are receiving this because you authored the thread. Reply to this email directly or view it on GitHub: https://github.com/milesgranger/pyrus-cramjam/issues/41#issuecomment-787436896

-- Sent from my Android device with K-9 Mail. Please excuse my brevity.

[milesgranger]

I was thinking that as well. Do you know how it works if it's translated from original C implementation? I've used c2rust before with various degrees of success. But not really knowledgeable if that avenue still causes licensing conflicts?

[milesgranger]

lzokay is a C++ implementation, released under MIT; suggesting, at least to me, it would be acceptable. :thinking:

[martindurant]

So long as we can blame someone else, then…

On Feb 28, 2021, at 14:15, Miles Granger notifications@github.com wrote:

lzokay is a C++ implementation, released under MIT; suggesting, at least to me, it would be acceptable. 🤔

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or unsubscribe.

[milesgranger]

I've successfully been able to port part of the C implementation to pure Rust, but with GPLv2 it seems very restrictive, covering even 'work derived'. But I see python-lzo to which fastparquet has an optional dependency is GPLv2, but fastparquet remains Apache 2. If this is acceptable, then I guess there isn't an issue for the rust implementation to have GPLv2 and cramjam to have MIT then, as it's basically the same relationship that fastparquet currently has, no? Maybe I'll just email the gentleman and see about at least a LGPL for the rust implementation. Worst case, cramjam is dual licensed under both depending on if lzo is included. :roll_eyes:

[martindurant]

There is a big difference between having an optional dependency on a library published and built by someone else, and directly embedding that library into your own code. That would make cramjam GPL, and I'm not sure about fastparquet (because cramjam would be a required dependency).

I cannot find a non-GPL LZO implementation, the original reference C impl was GPL. Maybe close this issue and ignore the problem.

[milesgranger]

Okay. Again, I'm a bit of a license novice here, just trying to understand; I was having ball with the rust implementation and would love to have it as a separate crate cramjam could make use of. And from what I understand, if I can at least publish a separate crate as LGPL, cramjam could link to it and maintain its current license; to do this, as I see it, would need permission from the author as I don't think GPLv2 can go directly LGPL. Feel free to close this issue if you'd like, but I'd still like to see if it can be done; by poking the author of lzo about it.

[martindurant]

If you link dynamically, as opposed to including in your own build, that's definitely OK.

would need permission from the author as I don't think GPLv2 can go directly LGPL

This I don't know.

I am just-above-novice...