xerial / snappy-java

Snappy compressor/decompressor for Java
Apache License 2.0
1.04k stars 232 forks source link

Snappy 1.1.7.1 and 1.1.7.2 does not work on ppc64le because of glibc 2.22 dependency #220

Closed nbathra closed 4 years ago

nbathra commented 6 years ago

java.lang.UnsatisfiedLinkError: /tmp/snappy-1.1.7-47312095-8ab9-48a7-885f-cea7c7614fd6-libsnappyjava.so: /lib64/ld64.so.2: version `GLIBC_2.22' not found (required by /tmp/snappy-1.1.7-47312095-8ab9-48a7-885f-cea7c7614fd6-libsnappyjava.so)

Any specific reason why this dependecy is introduced in the first place? Secondly, I couldn't find the GlibC 2.22 for ppc64le for RHEL74 and RHEL75 , in that case can snappy be compiled with GLIB2.17 itself ?

xerial commented 6 years ago

@nbathra Unless we build snappy-java using static link, dependencies to glibc will be included. To do so, we usually need a custom g++.

ppc64le binary is built by @odaira, and I think his OS's glibc version is newer than yours.

For example, linux, x86_64 native libraries is built using a custom g++ to do static link to embed libc++ (https://github.com/xerial/snappy-java/blob/master/BUILD.md#building-linux-x86_64-binary)

If you build your own snappy-java, using make native command, it should work even in RHEL74. Generally speaking, to build snappy-java for supporting older glibc versions, we need to use really old versions of Linux.

jamesbroadhead commented 6 years ago

@xerial we have run into this as well. We need to support RHEL7 on PPC - and RHEL7's glibc version is glibc 2.17

+ ldd ./src/main/resources/org/xerial/snappy/native/Linux/ppc64le/libsnappyjava.so
./src/main/resources/org/xerial/snappy/native/Linux/ppc64le/libsnappyjava.so: /lib64/ld64.so.2: version `GLIBC_2.22' not found (required by ./src/main/resources/org/xerial/snappy/native/Linux/ppc64le/libsnappyjava.so)
    linux-vdso64.so.1 =>  (0x00003fff7d300000)
    libm.so.6 => /lib64/power8/libm.so.6 (0x00003fff7d050000)
    libc.so.6 => /lib64/power8/libc.so.6 (0x00003fff7ce70000)
    /lib64/ld64.so.2 (0x0000000047020000)

This is not a problem with snappy-java's builds on x86 - these work fine on RHEL7 - so they must be built against an older glibc than that used on ppc64le.

Could you please reconsider & do a build and release which is compatible with RHEL on ppc64le?

We would really prefer not to maintain a forked build - but could provide a build of libsnappyjava.so built on RHEL7/ppc64le if you would consider releasing it.

edit: Please note that the prior ppc64le build worked on RHEL7 --

#!/bin/bash

set -x

uname -a

lsb_release --all

git log -n 3 --date=iso ./src/main/resources/org/xerial/snappy/native/Linux/ppc64le/libsnappyjava.so

echo "The current build"
git checkout 13299c1fda0df153d5ec651f829e861d28d2b9be
ldd ./src/main/resources/org/xerial/snappy/native/Linux/ppc64le/libsnappyjava.so

echo "The prior build"
git checkout 4ff7447abf8a5fa24c366e7637bd44e597a60c47
ldd ./src/main/resources/org/xerial/snappy/native/Linux/ppc64le/libsnappyjava.so
$ ./do
+ uname -a
Linux rhel71-ppc-dev.pic.build.10gen.cc 3.10.0-229.ael7b.ppc64le #1 SMP Fri Jan 30 12:03:50 EST 2015 ppc64le ppc64le ppc64le GNU/Linux
+ lsb_release --all
LSB Version:    :core-4.1-noarch:core-4.1-ppc64le
Distributor ID: RedHatEnterpriseServer
Description:    Red Hat Enterprise Linux Server release 7.1 (Maipo)
Release:    7.1
Codename:   Maipo
+ git log -n 3 --date=iso ./src/main/resources/org/xerial/snappy/native/Linux/ppc64le/libsnappyjava.so
commit 13299c1fda0df153d5ec651f829e861d28d2b9be
Author: Rei Odaira <rodaira@us.ibm.com>
Date:   2017-11-16 13:15:41 -0600

    Update Linux/ppc64le/libsnappyjava.so

commit 4ff7447abf8a5fa24c366e7637bd44e597a60c47
Author: Taro L. Saito <leo@xerial.org>
Date:   2017-11-14 11:37:04 -0800

    Add native libs excpet win and freebsd

commit e20494cfb29d562e84eda24c02aab2a15d5f4e2d
Author: Taro L. Saito <leo@xerial.org>
Date:   2017-02-14 22:26:06 -0800

    Remove ppc64. Rebuild ppc64le
+ echo 'The current build'
The current build
+ git checkout 13299c1fda0df153d5ec651f829e861d28d2b9be
Note: checking out '13299c1fda0df153d5ec651f829e861d28d2b9be'.

You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by performing another checkout.

If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -b with the checkout command again. Example:

  git checkout -b new_branch_name

HEAD is now at 13299c1... Update Linux/ppc64le/libsnappyjava.so
+ ldd ./src/main/resources/org/xerial/snappy/native/Linux/ppc64le/libsnappyjava.so
./src/main/resources/org/xerial/snappy/native/Linux/ppc64le/libsnappyjava.so: /lib64/ld64.so.2: version `GLIBC_2.22' not found (required by ./src/main/resources/org/xerial/snappy/native/Linux/ppc64le/libsnappyjava.so)
    linux-vdso64.so.1 =>  (0x00003fff7d300000)
    libm.so.6 => /lib64/power8/libm.so.6 (0x00003fff7d050000)
    libc.so.6 => /lib64/power8/libc.so.6 (0x00003fff7ce70000)
    /lib64/ld64.so.2 (0x0000000047020000)
+ echo 'The prior build'
The prior build
+ git checkout 4ff7447abf8a5fa24c366e7637bd44e597a60c47
Previous HEAD position was 13299c1... Update Linux/ppc64le/libsnappyjava.so
HEAD is now at 4ff7447... Add native libs excpet win and freebsd
+ ldd ./src/main/resources/org/xerial/snappy/native/Linux/ppc64le/libsnappyjava.so
    linux-vdso64.so.1 =>  (0x00003fff937c0000)
    libm.so.6 => /lib64/power8/libm.so.6 (0x00003fff935c0000)
    libc.so.6 => /lib64/power8/libc.so.6 (0x00003fff933e0000)
    /lib64/ld64.so.2 (0x000000003f750000)
xerial commented 6 years ago

@jamesbroadhead Binaries for Linux/ppc64le can be built by using a cross compiler provided by https://github.com/dockcross/dockcross/tree/master/linux-ppc64le

If we can create a docker image that contains a cross compiler for ppc64le and uses an older version of glibc (e.g., REHL7), we can build a binary for your environment by simply changing this line: https://github.com/xerial/snappy-java/blob/master/Makefile#L204

xerial commented 4 years ago

It should be fixed in snappy-java 1.1.7.4

nbathra commented 4 years ago

Thanks @xerial .