search

lz4 / lz4-java

LZ4 compression for Java
Apache License 2.0
1.1k stars 253 forks source link

Compile against older glibc to add support for CentOS 6 #163

Closed dcapwell closed 4 years ago

dcapwell commented 4 years ago

For the purposes of Apache licensing and copyright, this reported issue is Not a Contribution.

The latest version currently requires GLIBC_2.14 but this is too new for some older environments such as CentOS 6 (as pointed out in https://github.com/lz4/lz4-java/issues/160); to aid in upgrading to the latest version it would be good to support 2.12.

This was found in Apache Cassandra where we were upgrading from 1.3 to 1.7

odaira commented 4 years ago

I built the JNI binding on CentOS 6 and confirmed that the minimum required version was glibc 2.2.5. However, I also found that the performance of the built JNI library was 27% slower than the latest lz4-java 1.7.1, probably because gcc in CentOS 6 was version 4.4.7, which is quite old. I am going to install a more recent version of gcc to CentOS 6 and will check the performance again.

odaira commented 4 years ago

I built the JNI binding on CentOS 6 with gcc 5.4.0 and confirmed that the throughput was the same as lz4-java-1.7.1. FYI, the one compiled with the latest gcc 9.2.0 was 5% slower, so I decided to use gcc 5.4.0, which is the same version as what I have been using.

dcapwell commented 4 years ago

Making sure I understand. Are you saying that native support won't happen for CentOS 6 environments or that the compiling with newer GCC was enough to get older environments to work with the latest gains?

odaira commented 4 years ago

The compiling with newer GCC was enough to get older environments to work with the latest gains. The master branch should run on CentOS 6 and still retains the same speed as before. The fix will be included in the next release.

dcapwell commented 4 years ago

Thats great, thanks!

dcapwell commented 3 years ago

The fix will be included in the next release.

Is there any update on a release? Apache Cassandra is at the RC stage for our 4.0 release, so it would be amazing if we can bump our lz4 version to work with OEL6 again.

Thanks @odaira !

odaira commented 3 years ago

OK, I'll release the next version by the end of this month.

dcapwell commented 3 years ago

Any update? Apache Cassandra is in-talks to release GA if no issues found in the next 2 weeks

odaira commented 3 years ago

Sorry, I ran into an issue at the last step in publishing a jar to the maven repo. I'll try to fix it this weekend.

odaira commented 3 years ago

Released 1.8.0.

dcapwell commented 3 years ago

Thanks so much!