search

kubo / ruby-oci8

Ruby-oci8 - Oracle interface for ruby
Other
169 stars 75 forks source link

Possible To Force Static Build? #232

Closed metaskills closed 3 years ago

metaskills commented 3 years ago

When installing ruby-oci8 I'd like to generate a static file vs "oci8lib_250.so". Do you have any tips on how to do this?

kubo commented 3 years ago

Could you explain a little more?

metaskills commented 3 years ago

Sure, I'd like to build a gem like this pre-compiled with static linking (vs /opt directory links) to package as an internal gem. https://github.com/customink/mysql2-lambda

kubo commented 3 years ago

Oracle doesn't provide static client libraries. So it is impossible.

However it is possible to bundle patched Oracle client libraries in a pre-compiled gem if the platform is Linux.

Prepare Oracle instant client and install patchelf in advance.

$ git clone https://github.com/kubo/ruby-oci8.git
$ cd ruby-oci8
$ git checkout ruby-oci8-2.2.9
$ export LD_LIBRARY_PATH=/path/to/oracle/instant/client
$ make
$ cp /path/to/oracle/instant/client/lib*.so* lib # copy all Oracle shared libraries.
$ patchelf --set-rpath '$ORIGIN' lib/libclntsh.so.*
$ patchelf --set-rpath '$ORIGIN' ext/oci8/oci8lib_???.so
$ vi ruby-oci8.gemspec # vi or any editor

Replace these three lines with the following one line.

    files += Dir.glob('lib/*.so*')
$ gem build ruby-oci8.gemspec -- current

This gem includes Oracle client libraries along with oci8lib_???.so. It works on any directory.

Well, I suspect that Oracle doesn't support patched instant client.

metaskills commented 3 years ago

Thanks!!! I had to make one small change:

patchelf --set-rpath '$ORIGIN/../../lib' ext/oci8/oci8lib_???.so

After that everything appeared in order after installing the gem again on a fresh container sans the /opt install.

ldd /var/lang/lib/ruby/gems/2.5.0/gems/ruby-oci8-2.2.9.0/ext/oci8/oci8lib_250.so
  linux-vdso.so.1 =>  (0x00007fff834a0000)
  libclntsh.so.11.1 => /var/lang/lib/ruby/gems/2.5.0/gems/ruby-oci8-2.2.9.0/ext/oci8/../../lib/libclntsh.so.11.1 (0x00007f1db3154000)
  libpthread.so.0 => /lib64/libpthread.so.0 (0x00007f1db2f38000)
  libgmp.so.10 => /usr/lib64/libgmp.so.10 (0x00007f1db2cc2000)
  libdl.so.2 => /lib64/libdl.so.2 (0x00007f1db2abe000)
  libcrypt.so.1 => /lib64/libcrypt.so.1 (0x00007f1db2887000)
  libm.so.6 => /lib64/libm.so.6 (0x00007f1db2585000)
  libc.so.6 => /lib64/libc.so.6 (0x00007f1db21b7000)
  /lib64/ld-linux-x86-64.so.2 (0x00007f1db5d82000)
  libnnz11.so => /var/lang/lib/ruby/gems/2.5.0/gems/ruby-oci8-2.2.9.0/ext/oci8/../../lib/libnnz11.so (0x00007f1db1dea000)
  libnsl.so.1 => /lib64/libnsl.so.1 (0x00007f1db1bd0000)
  libaio.so.1 => /lib64/libaio.so.1 (0x00007f1db19cf000)
  libfreebl3.so => /lib64/libfreebl3.so (0x00007f1db17cd000)

ldd /var/lang/lib/ruby/gems/2.5.0/gems/ruby-oci8-2.2.9.0/lib/libnnz11.so
  linux-vdso.so.1 =>  (0x00007fffdf7f9000)
  libc.so.6 => /lib64/libc.so.6 (0x00007f98d1268000)
  /lib64/ld-linux-x86-64.so.2 (0x00007f98d1a03000)

ldd /var/lang/lib/ruby/gems/2.5.0/gems/ruby-oci8-2.2.9.0/lib/libclntsh.so.11.1
  linux-vdso.so.1 =>  (0x00007ffe1dbfe000)
  libnnz11.so => /var/lang/lib/ruby/gems/2.5.0/gems/ruby-oci8-2.2.9.0/lib/libnnz11.so (0x00007f92df0a9000)
  libdl.so.2 => /lib64/libdl.so.2 (0x00007f92deea5000)
  libm.so.6 => /lib64/libm.so.6 (0x00007f92deba3000)
  libpthread.so.0 => /lib64/libpthread.so.0 (0x00007f92de987000)
  libnsl.so.1 => /lib64/libnsl.so.1 (0x00007f92de76d000)
  libc.so.6 => /lib64/libc.so.6 (0x00007f92de39f000)
  libaio.so.1 => /lib64/libaio.so.1 (0x00007f92de19e000)
  /lib64/ld-linux-x86-64.so.2 (0x00007f92e1e6b000)

However, here is what I got when doing a require in IRB.

require 'ruby-oci8'
Traceback (most recent call last):
       11: from /var/lang/bin/irb:11:in `<main>'
       10: from (irb):1
        9: from /var/lang/lib/ruby/site_ruby/2.5.0/rubygems/core_ext/kernel_require.rb:37:in `require'
        8: from /var/lang/lib/ruby/site_ruby/2.5.0/rubygems/core_ext/kernel_require.rb:160:in `rescue in require'
        7: from /var/lang/lib/ruby/site_ruby/2.5.0/rubygems/core_ext/kernel_require.rb:160:in `require'
        6: from /var/lang/lib/ruby/gems/2.5.0/gems/ruby-oci8-2.2.9.0/lib/ruby-oci8.rb:1:in `<top (required)>'
        5: from /var/lang/lib/ruby/site_ruby/2.5.0/rubygems/core_ext/kernel_require.rb:85:in `require'
        4: from /var/lang/lib/ruby/site_ruby/2.5.0/rubygems/core_ext/kernel_require.rb:85:in `require'
        3: from /var/lang/lib/ruby/gems/2.5.0/gems/ruby-oci8-2.2.9.0/lib/oci8.rb:112:in `<top (required)>'
        2: from /var/lang/lib/ruby/site_ruby/2.5.0/rubygems/core_ext/kernel_require.rb:85:in `require'
        1: from /var/lang/lib/ruby/site_ruby/2.5.0/rubygems/core_ext/kernel_require.rb:85:in `require'
kubo commented 3 years ago

Thanks!!! I had to make one small change:

patchelf --set-rpath '$ORIGIN/../../lib' ext/oci8/oci8lib_???.so

How did you package the pre-compiled gem? gem build ruby-oci8.gemspec -- current copies ext/oci8/oci8lib_???.so to lib. So it finds Oracle libraries in $ORIGIN, not in $ORIGIN/../../lib.

However, here is what I got when doing a require in IRB.

Are there no lines after the traceback? It usually prints an error message after that. I guess that the message is LoadError (cannot load such file -- oci8lib_250.so). If so, you need to copy the file to lib.

metaskills commented 3 years ago

gem build ruby-oci8.gemspec -- current copies ext/oci8/oci8lib_???.so to lib

It does not tho. I've tried it a few times.

I guess that the message is LoadError (cannot load such file

Nope, that's the whole error right after the require.

metaskills commented 3 years ago

Sorry, darn terminal colors, it was LoadError (cannot load such file -- oci8lib_250)

kubo commented 3 years ago

gem build ruby-oci8.gemspec -- current copies ext/oci8/oci8lib_???.so to lib

It does not tho. I've tried it a few times.

oci8lib_???.so is copied here. Could you post your ruby-oci8.gemspec at gist and post the link?

metaskills commented 3 years ago

OK this was really helpful. Much like the MySQL project example I have a copy of the gem spec that wraps the gem with additional version changes. So that is why I did not have the oci8lib_???.so being copied. I did that in my Dockerfile and simplified the patchelf commands. EVERYTHING WORKS!

Here is my final solution with some things missing as part of my project's build scripts. I'm expecting not to be able to post these to Rubygems since it includes Oracle stuff. But it does work! The container still has to install libaio and I guess I could package that too but this is good for now. Thanks again for your help!

FROM amazon/aws-sam-cli-build-image-ruby2.7

WORKDIR /build

ARG RUBY_VERSION
ARG RUBY_OCI8_VERSION
ARG RUBY_OCI8_BUILD_VERSION
ENV RUBY_OCI8_VERSION=$RUBY_OCI8_VERSION
ENV RUBY_OCI8_BUILD_VERSION=$RUBY_OCI8_BUILD_VERSION

RUN echo '== Oracle Instant Client Files =='
COPY oracle-instant-client-zip .
RUN yum install -y libaio
RUN mkdir -p /opt/oracle \
    && cd /opt/oracle \
    && unzip /build/instantclient-basiclite-linux.x64-11.2.0.4.0.zip \
    && unzip /build/instantclient-sdk-linux.x64-11.2.0.4.0.zip \
    && unzip /build/instantclient-sqlplus-linux.x64-11.2.0.4.0.zip \
    && cd /opt/oracle/instantclient_11_2 \
    && ln -s libclntsh.so.11.1 libclntsh.so \
    && ln -s libocci.so.11.1 libocci.so \
    && echo '/opt/oracle/instantclient_11_2' > /etc/ld.so.conf.d/oracle-instantclient.conf \
    && /sbin/ldconfig \
    && echo 'export PATH="$PATH:/opt/oracle/instantclient_11_2"' >> /etc/profile.d/oracle.sh
ENV PATH=${PATH}:/opt/oracle/instantclient_11_2

RUN echo '== Install patchelf =='
RUN git clone https://github.com/NixOS/patchelf.git \
    && cd ./patchelf \
    && git checkout 0.11 \
    && ./bootstrap.sh \
    && ./configure --prefix=/opt \
    && make \
    && make install

RUN echo '== Install ruby-oci8 Gem and Patch =='
RUN git clone https://github.com/kubo/ruby-oci8.git \
    && cd ./ruby-oci8 \
    && git checkout "ruby-oci8-${RUBY_OCI8_VERSION}" \
    && export LD_LIBRARY_PATH=/opt/oracle/instantclient_11_2 \
    && make \
    && cd ./lib \
    && cp /opt/oracle/instantclient_11_2/lib*.so* . \
    && rm -rf libclntsh.so \
    && rm -rf libocci.so \
    && ln -s libclntsh.so.11.1 libclntsh.so \
    && ln -s libocci.so.11.1 libocci.so \
    && cd .. \
    && mv ./ext/oci8/oci8lib_???.so ./lib \
    && patchelf --set-rpath '$ORIGIN' lib/libclntsh.so \
    && patchelf --set-rpath '$ORIGIN' lib/oci8lib_???.so

RUN echo '== Build Gem File =='
COPY VERSION /build/ruby-oci8
COPY BUILD /build/ruby-oci8
COPY ruby-oci8.gemspec /build/ruby-oci8
RUN cd ./ruby-oci8 \
    && find -path './.*' -delete \
    && echo "$RUBY_VERSION" > RUBY_VERSION \
    && gem build ruby-oci8.gemspec
$ RUBY_VERSION=27 ./bin/build
IMAGE_NAME="ruby-oci8-lambda-${RUBY_VERSION}"
RUBY_OCI8_VERSION=$(cat VERSION)
RUBY_OCI8_BUILD_VERSION=$(cat BUILD)

docker build \
  --no-cache \
  --build-arg "DOCKER_FROM=amazon/aws-sam-cli-build-image-ruby${RUBY_VERSION}"  \
  --build-arg "RUBY_VERSION=${RUBY_VERSION}" \
  --build-arg "RUBY_OCI8_VERSION=${RUBY_OCI8_VERSION}" \
  --build-arg "RUBY_OCI8_BUILD_VERSION=${RUBY_OCI8_BUILD_VERSION}" \
  --tag $IMAGE_NAME \
  --file "Dockerfile-${RUBY_VERSION}" \
  .

docker run \
  --rm \
  --volume "${PWD}:/var/task" \
  "${IMAGE_NAME}:latest" \
  sh -c "cp /build/ruby-oci8/ruby-oci8-${RUBY_OCI8_VERSION}.${RUBY_OCI8_BUILD_VERSION}.${RUBY_VERSION}.gem /var/task"
metaskills commented 3 years ago

( edited above )