JVM crashes if snappy is not in system library path

[GoogleCodeExporter]

Loading the snappy library via System.loadLibrary() succeeds if the library is 
in the Hadoop lib/native directory (and thus in the java.library.path).  
However, the dlopen() call in the C code fails if the library isn't in the 
system library path.  The Java code ignores this error and goes on to use the 
library, which then crashes the JVM.

This patch does three things:

1) Use RTLD_DEFAULT instead of dlopen(), as the snappy library is already 
loaded via System.loadLibrary().
2) Initialize the native code in LoadSnappy, allowing the entire loading 
process can be guarded by isLoaded().
3) Add isLoaded() guards to the SnappyCompressor and SnappyDecompressor 
constructors, preventing the JVM from crashing if they are used directly.

Original issue reported on code.google.com by electrum on 24 Aug 2011 at 7:11

Attachments:

• hadoop-snappy-dlopen.patch

[GoogleCodeExporter]

Original comment by electrum on 24 Aug 2011 at 9:17

Attachments:

• hadoop-snappy-dlopen.patch

[GoogleCodeExporter]

I tried this patch with hadoop-snappy trunk (and google snappy trunk ) and it 
fails : 
I'm using :

Ubuntu 10.04 64 bits x86_64 
2.6.32-39-generic #86-Ubuntu SMP
gcc (Ubuntu 4.4.3-4ubuntu5.1) 4.4.3
Apache Maven 2.2.1 (rdebian-1)
Java version: 1.6.0_21
Java home: /usr/lib/jvm/java-6-sun-1.6.0.21/jre

Error in mvn package : 

[exec] config.status: creating Makefile
     [exec] config.status: creating config.h
     [exec] config.status: executing depfiles commands
     [exec] config.status: executing libtool commands
     [exec] depbase=`echo src/org/apache/hadoop/io/compress/snappy/SnappyCompressor.lo | sed 's|[^/]*$|.deps/&|;s|\.lo$||'`;\
     [exec]     /bin/bash ./libtool --tag=CC   --mode=compile gcc -DHAVE_CONFIG_H -I.  -I/usr/lib/jvm/java-6-sun/include -I/usr/lib/jvm/java-6-sun/include/linux -I/home/users/eric/hadoop-snappy-read-only/src/main/native/src -Isrc/org/apache/hadoop/io/compress/snappy -I/usr/local/include -g -Wall -fPIC -O2 -m64 -g -O2 -MT src/org/apache/hadoop/io/compress/snappy/SnappyCompressor.lo -MD -MP -MF $depbase.Tpo -c -o src/org/apache/hadoop/io/compress/snappy/SnappyCompressor.lo src/org/apache/hadoop/io/compress/snappy/SnappyCompressor.c &&\
     [exec]     src/org/apache/hadoop/io/compress/snappy/SnappyCompressor.c: In function ‘Java_org_apache_hadoop_io_compress_snappy_SnappyCompressor_initIDs’:
     [exec] src/org/apache/hadoop/io/compress/snappy/SnappyCompressor.c:65: error: ‘RTLD_DEFAULT’ undeclared (first use in this function)
     [exec] src/org/apache/hadoop/io/compress/snappy/SnappyCompressor.c:65: error: (Each undeclared identifier is reported only once
     [exec] src/org/apache/hadoop/io/compress/snappy/SnappyCompressor.c:65: error: for each function it appears in.)
     [exec] src/org/apache/hadoop/io/compress/snappy/SnappyCompressor.c: In function ‘Java_org_apache_hadoop_io_compress_snappy_SnappyCompressor_compressBytesDirect’:
     [exec] src/org/apache/hadoop/io/compress/snappy/SnappyCompressor.c:108: warning: passing argument 4 of ‘dlsym_snappy_compress’ from incompatible pointer type
     [exec] src/org/apache/hadoop/io/compress/snappy/SnappyCompressor.c:108: note: expected ‘size_t *’ but argument is of type ‘jint *’
     [exec] mv -f $depbase.Tpo $depbase.Plo
     [exec] libtool: compile:  gcc -DHAVE_CONFIG_H -I. -I/usr/lib/jvm/java-6-sun/include -I/usr/lib/jvm/java-6-sun/include/linux -I/home/users/eric/hadoop-snappy-read-only/src/main/native/src -Isrc/org/apache/hadoop/io/compress/snappy -I/usr/local/include -g -Wall -fPIC -O2 -m64 -g -O2 -MT src/org/apache/hadoop/io/compress/snappy/SnappyCompressor.lo -MD -MP -MF src/org/apache/hadoop/io/compress/snappy/.deps/SnappyCompressor.Tpo -c src/org/apache/hadoop/io/compress/snappy/SnappyCompressor.c  -fPIC -DPIC -o src/org/apache/hadoop/io/compress/snappy/.libs/SnappyCompressor.o
     [exec] make: *** [src/org/apache/hadoop/io/compress/snappy/SnappyCompressor.lo] Error 1
[INFO] ------------------------------------------------------------------------
[ERROR] BUILD ERROR
[INFO] ------------------------------------------------------------------------
[INFO] An Ant BuildException has occured: The following error occurred while 
executing this line:
/home/users/eric/hadoop-snappy-read-only/maven/build-compilenative.xml:75: exec 
returned: 2

Original comment by epalac...@datasalt.com on 21 Mar 2012 at 11:06