search

ibmdb / node-ibm_db

IBM DB2 and IBM Informix bindings for node
MIT License
190 stars 151 forks source link

Alpine support - musl libc #217

Closed willfarrell closed 7 years ago

willfarrell commented 7 years ago

I'm getting the following error while attempting to run the example on alpine.

/var/www/node_modules/bindings/bindings.js:83
        throw e
        ^

Error: Error relocating /var/www/node_modules/ibm_db/build/Release/../../installer/clidriver/lib/libdb2.so.1: pthread_mutexattr_setkind_np: symbol not found
    at Object.Module._extensions..node (module.js:598:18)
    at Module.load (module.js:488:32)
    at tryModuleLoad (module.js:447:12)
    at Function.Module._load (module.js:439:3)
    at Module.require (module.js:498:17)
    at require (internal/module.js:20:19)
    at bindings (/var/www/node_modules/bindings/bindings.js:76:44)
    at Object.<anonymous> (/var/www/node_modules/ibm_db/lib/odbc.js:27:31)
    at Module._compile (module.js:571:32)
    at Object.Module._extensions..js (module.js:580:10)

The file does exist at: /var/www/node_modules/ibm_db/installer/clidriver/lib/libdb2.so.1.

Source: Dockerfile

FROM node:7-alpine

RUN apk add --no-cache --virtual .build-deps \
    python make g++

RUN mkdir /var/www \
    && cd /var/www \
    && npm install ibm_db

RUN apk del .build-deps

RUN apk add --no-cache --virtual .odbc-rundeps \
        libgcrypt \
        linux-pam \
        libxml2 \
    && ln -s -f /usr/lib/libgcrypt.so.20 /usr/lib/libcrypt.so.1

COPY ./app.js /var/www/

CMD [ "node", "/var/www/app.js" ]

app.js

var ibmdb = require('ibm_db');

ibmdb.open("DATABASE=Test;HOSTNAME=*******;UID=******;PWD=******;PORT=446;PROTOCOL=TCPIP", function (err,conn) {
    if (err) return console.log(err);

    conn.query('***********', function (err, data) {
        if (err) console.log(err);
        else console.log(data);

        conn.close(function () {
            console.log('done');
        });
    });
});
bimalkjha commented 7 years ago

Ok. So, the problem is this error:

Error: Error relocating /var/www/node_modules/ibm_db/build/Release/../../installer/clidriver/lib/libdb2.so.1: pthread_mutexattr_setkind_np: symbol not found
    at Object.Module._extensions..node (module.js:598:18)

While loading libdb2.so.1, OS does not find the symbol pthread_mutexattr_setkind_np. I searched for it on my linux system and found that pthread_mutexattr_setkind_np should be in glibc library:

$ nm libdb2.so.1 | grep pthread_mutexattr_setkind_np U pthread_mutexattr_setkind_np@@GLIBC_2.2.5

Please share output of ldd libdb2.so.1 command. Also, node -v result. Thanks.

willfarrell commented 7 years ago

I added in glibc to see if that solved it. Same result.

For those looking for how to do it:

RUN cd /tmp \
    && wget -q -O /etc/apk/keys/sgerrand.rsa.pub https://raw.githubusercontent.com/sgerrand/alpine-pkg-glibc/master/sgerrand.rsa.pub \
    && wget https://github.com/sgerrand/alpine-pkg-glibc/releases/download/2.23-r3/glibc-2.23-r3.apk \
    && apk add --no-cache glibc-2.23-r3.apk

NodeJS: v7.4.0 npm: v4.0.5

$ docker run odbc ldd /var/www/node_modules/ibm_db/installer/clidriver/lib/libdb2.so.1
Error relocating /var/www/node_modules/ibm_db/installer/clidriver/lib/libdb2.so.1: pthread_mutexattr_setkind_np: symbol not found
Error relocating /var/www/node_modules/ibm_db/installer/clidriver/lib/libdb2.so.1: __strtof_internal: symbol not found
Error relocating /var/www/node_modules/ibm_db/installer/clidriver/lib/libdb2.so.1: pthread_attr_setaffinity_np: symbol not found
Error relocating /var/www/node_modules/ibm_db/installer/clidriver/lib/libdb2.so.1: __register_atfork: symbol not found
Error relocating /var/www/node_modules/ibm_db/installer/clidriver/lib/libdb2.so.1: __strtod_internal: symbol not found
Error relocating /var/www/node_modules/ibm_db/installer/clidriver/lib/libdb2.so.1: sysctl: symbol not found
Error relocating /var/www/node_modules/ibm_db/installer/clidriver/lib/libdb2.so.1: backtrace: symbol not found
Error relocating /var/www/node_modules/ibm_db/installer/clidriver/lib/libdb2.so.1: getgrent_r: symbol not found
Error relocating /var/www/node_modules/ibm_db/installer/clidriver/lib/libdb2.so.1: __res_init: symbol not found
Error relocating /var/www/node_modules/ibm_db/installer/clidriver/lib/libdb2.so.1: dlvsym: symbol not found
    ldd (0x5608749ab000)
    libcrypt.so.1 => /usr/lib/libcrypt.so.1 (0x7f336255c000)
    libdl.so.2 => ldd (0x5608749ab000)
    libpthread.so.0 => ldd (0x5608749ab000)
    librt.so.1 => ldd (0x5608749ab000)
    libpam.so.0 => /lib/libpam.so.0 (0x7f3362350000)
    libxml2.so.2 => /usr/lib/libxml2.so.2 (0x7f3362025000)
    libm.so.6 => ldd (0x5608749ab000)
    libstdc++.so.6 => /usr/lib/libstdc++.so.6 (0x7f3361cd5000)
    libgcc_s.so.1 => /usr/lib/libgcc_s.so.1 (0x7f3361ac2000)
    libc.so.6 => ldd (0x5608749ab000)
    libgpg-error.so.0 => /usr/lib/libgpg-error.so.0 (0x7f33618b1000)
    libz.so.1 => /lib/libz.so.1 (0x7f336169b000)
bimalkjha commented 7 years ago

So, your alpine OS has below missing libraries:

libdl.so.2 => ldd (0x5608749ab000)
libpthread.so.0 => ldd (0x5608749ab000)
librt.so.1 => ldd (0x5608749ab000)
libm.so.6 => ldd (0x5608749ab000)
libc.so.6 => ldd (0x5608749ab000)

On my Linux system I can see these libs under /lib64:

        libdl.so.2 => /lib64/libdl.so.2 (0x00002aaaae2c9000)
        libpthread.so.0 => /lib64/libpthread.so.0 (0x00002aaaae3cd000)
        librt.so.1 => /lib64/librt.so.1 (0x00002aaaae4e7000)
        libm.so.6 => /lib64/libm.so.6 (0x00002aaaae940000)
        libc.so.6 => /lib64/libc.so.6 (0x00002aaaaeda1000)

Please contact docker team and get these libs. Now, I am curious to know - after installing ibm_db, which server you want to connect? None of the IBM DB2 or DashDB driver would work on this system with missing libraries. It is not an issue with node-ibm_db driver. Thanks.

willfarrell commented 7 years ago

Looks like it's related to this: https://github.com/gliderlabs/docker-alpine/blob/master/docs/caveats.md#incompatible-binaries

I'll dig deeper and post a solution when I find one. Thank you for the assistance.

willfarrell commented 7 years ago

For other running into this. Checkout: https://github.com/gliderlabs/docker-alpine/issues/11#issuecomment-161710297 and https://github.com/frol/docker-alpine-glibc/blob/master/Dockerfile

One of two things needs to happen for this to work.

  1. compile your own version of node on glibc
  2. compile imb_db2 deps for musl libc

The later seems like the better approach, but could take awhile to figure out how to do it.

willfarrell commented 7 years ago

@hwangbible do you remember how you resolve your issue at https://github.com/gliderlabs/docker-alpine/issues/11#issuecomment-161710297 ?

bimalkjha commented 7 years ago

@willfarrell I would suggest to install musl libc library and check file /usr/lib/libc.so.6 is available or not. If libc.so.6 is available, then run ldd libdb2.so.1 and see it is able to find the library or not. Thanks.

willfarrell commented 7 years ago

musl libc is install by default on alpine. Which is giving us what we've been seeing above.

$ ldd /var/www/node_modules/ibm_db/installer/clidriver/lib/libdb2.so.1
ldd (0x5630cd73b000)
    libcrypt.so.1 => /usr/lib/libcrypt.so.1 (0x7f991cd7f000)
    libdl.so.2 => ldd (0x5630cd73b000)
    libpthread.so.0 => ldd (0x5630cd73b000)
    librt.so.1 => ldd (0x5630cd73b000)
    libpam.so.0 => /lib/libpam.so.0 (0x7f991cb73000)
    libxml2.so.2 => /usr/lib/libxml2.so.2 (0x7f991c848000)
    libm.so.6 => ldd (0x5630cd73b000)
    libstdc++.so.6 => /usr/lib/libstdc++.so.6 (0x7f991c4f8000)
    libgcc_s.so.1 => /usr/lib/libgcc_s.so.1 (0x7f991c2e5000)
    libc.so.6 => ldd (0x5630cd73b000)
    libgpg-error.so.0 => /usr/lib/libgpg-error.so.0 (0x7f991c0d4000)
    libz.so.1 => /lib/libz.so.1 (0x7f991bebe000)

For what I'm understanding, ldd on alpine is built on musl libc and thus is unable to findlibc.so.6 that is now included w/ glibc. There is a seperate ldd included with the glibc package however it doesn't support dynamic libraries (it's a sh script). Is it possible to have IBM compile the binary against musl libc? Alpine is a growing environment with the increasing use of smaller and smaller docker containers.

bimalkjha commented 7 years ago

Is it possible to have IBM compile the binary against musl libc? --> There is no such plan as of now. Thanks.

bimalkjha commented 7 years ago

Closing the issue as IBM ODBC/CLI driver is not supported on Alpine Linux and hence ibm_db can not work on Alpine. Checked with Kelly Rodger at IBM and he confirmed that Alpine Linux is not a supported platform for DB2. Thanks.

kentonbmax commented 6 years ago

If not alpine what is a super lightweight linux container docker image to use?

bimalkjha commented 6 years ago

@kentonbmax You can use Ubuntu docker image. Thanks.

kentonbmax commented 6 years ago

Hey, I had difficulty with the ubuntu image and tried the oracle linux image (smaller). it works but am getting license issue that I commented on in a different issue.

bimalkjha commented 6 years ago

@kentonbmax Could you please paste the error about license issue here? that would help to understand the problem. Also, let me know your database is on which OS? License file is required only if your DB2 server is installed on z/OS or i-Series systems. Thanks.

kentonbmax commented 6 years ago

DB Server: i-series. Container: Oracle Linux { Error: [IBM][CLI Driver] SQL1598N An attempt to connect to the database server failed because of a licensing problem. SQLSTATE=42968

errors: [], error: '[node-odbc] SQL_ERROR', message: '[IBM][CLI Driver] SQL1598N An attempt to connect to the database server failed because of a licensing problem. SQLSTATE=42968\n', state: '42968' }

bimalkjha commented 6 years ago

@kentonbmax You need to copy db2connect license file under ibm_db/installer/clidriver/license folder to access i-Series server using ibm_db. Please contact your i-series system adminstrator or IBM support to get the db2connect license file. The license file name should be like db2con*.lic . Thanks.

kentonbmax commented 6 years ago

Hello, I copied my lic file to ./node_modules/ibm_db/installer/clidriver/license. This is the same license we use with windows and the .net core nuget package. Same error. On Thu, May 24, 2018 at 11:59 AM Bimal Kumar Jha notifications@github.com wrote:

@kentonbmax https://github.com/kentonbmax You need to copy db2connect license file under ibm_db/installer/clidriver/license folder to access i-Series server using ibm_db. Please contact your i-series system adminstrator or IBM support to get the db2connect license file. The license file name should be like db2con*.lic . Thanks.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/ibmdb/node-ibm_db/issues/217#issuecomment-391768031, or mute the thread https://github.com/notifications/unsubscribe-auth/APEoRImSer-fxykAtaPwkxwphko900pLks5t1tjagaJpZM4LlHty .

bimalkjha commented 6 years ago

@kentonbmax First check the db2level output of your windows system where this .lic file was working. If it is not DB2 V10.5, please get a db2 v10.5 license file. Second, delete existing clidriver/cfgcache/conlic.bin file and then try connection if you have the correct license file. On windows, we can find cfgcache folder under C:\ProgramData\IBM\DB2\ folder. If you are still facing issue, please update the connection string in ibm_db\installer\testODBCConnection.sh file near line no 29 for db2cli validate command and execute testODBCConnection.sh file. If it fails, share the generated files for investigation. Thanks.

kentonbmax commented 6 years ago

My .lic says I have v. 11.1. This file is the same that gets copied with our .net core nuget setup to production. We setup a env variable for that environment. Is setting up an env var in linux necessary?

On Fri, Jun 1, 2018 at 3:01 AM Bimal Kumar Jha notifications@github.com wrote:

@kentonbmax https://github.com/kentonbmax First check the db2level output of your windows system where this .lic file was working. If it is not DB2 V10.5, please get a db2 v10.5 license file. Second, delete existing clidriver/cfgcache/conlic.bin file and then try connection if you have the correct license file. On windows, we can find cfgcache folder under C:\ProgramData\IBM\DB2 folder. If you are still facing issue, please update the connection string in ibm_db\installer\testODBCConnection.sh file near line no 29 for db2cli validate command and execute testODBCConnection.sh file. If it fails, share the generated files for investigation. Thanks.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/ibmdb/node-ibm_db/issues/217#issuecomment-393783052, or mute the thread https://github.com/notifications/unsubscribe-auth/APEoREwPUV7QVw6ZU_G1dqPzL0AdlAvNks5t4ObjgaJpZM4LlHty .

bimalkjha commented 6 years ago

@kentonbmax If you already have Db2 V11.1 installed in your system, please share the output of "db2level" command. The v11.1 license file will not work with ibm_db, but you can tell ibm_db to use pre-installed db2 v11.1 by setting IBM_DB_HOME env variable. Setting up the env var in linux is not necessary. Thanks.

kentonbmax commented 6 years ago

We are at O/S 7.2 with database group 21. Output: DB21085I This instance or install (instance name, where applicable: "*") uses "64" bits and DB2 code release "SQL11012" with level identifier "0203010F". Informational tokens are "DB2 v11.1.2020.1393", "s1706091900", "DYN1701310100WIN64", and Fix Pack "2".

On Mon, Jun 4, 2018 at 10:13 AM Bimal Kumar Jha notifications@github.com wrote:

@kentonbmax https://github.com/kentonbmax If you already have Db2 V11.1 installed in your system, please share the output of "db2level" command. The v11.1 license file will not work with ibm_db, but you can tell ibm_db to use pre-installed db2 v11.1 by setting IBM_DB_HOME env variable. Setting up the env var in linux is not necessary. Thanks.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/ibmdb/node-ibm_db/issues/217#issuecomment-394369211, or mute the thread https://github.com/notifications/unsubscribe-auth/APEoRK4W801n0LMsTZGQ57sONniAtrVjks5t5UCIgaJpZM4LlHty .

kentonbmax commented 6 years ago

Also tried this approach using oracle/linux. I also copied the .lic to the /clidriver/license folder. The build is successful from the npm install but I get the same error on connect with or without the .lic file. Do you maybe have a linux image on Dockerhub that has the driver already setup?

RUN mkdir /clidriver

COPY ibm_data_server_driver_for_odbc_cli_linuxx64_v11.1.tar.gz /clidriver

RUN tar -xzf /clidriver/ibm_data_server_driver_for_odbc_cli_linuxx64_v11.1.tar.gz ENV IBM_DB_HOME "/clidriver"

On Mon, Jun 4, 2018 at 10:33 AM Kenton Bocock kentonbo@gmail.com wrote:

We are at O/S 7.2 with database group 21. Output: DB21085I This instance or install (instance name, where applicable: "*") uses "64" bits and DB2 code release "SQL11012" with level identifier "0203010F". Informational tokens are "DB2 v11.1.2020.1393", "s1706091900", "DYN1701310100WIN64", and Fix Pack "2".

On Mon, Jun 4, 2018 at 10:13 AM Bimal Kumar Jha notifications@github.com wrote:

@kentonbmax https://github.com/kentonbmax If you already have Db2 V11.1 installed in your system, please share the output of "db2level" command. The v11.1 license file will not work with ibm_db, but you can tell ibm_db to use pre-installed db2 v11.1 by setting IBM_DB_HOME env variable. Setting up the env var in linux is not necessary. Thanks.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/ibmdb/node-ibm_db/issues/217#issuecomment-394369211, or mute the thread https://github.com/notifications/unsubscribe-auth/APEoRK4W801n0LMsTZGQ57sONniAtrVjks5t5UCIgaJpZM4LlHty .

bimalkjha commented 6 years ago

@kentonbmax Please note that the license file that you have is for db2 v11.1 and it will not work with ibm_db which uses DB2 v10.5. You need a db2 v10.5 db2connect license file for ibm_db.

The ibm_data_server_driver_for_odbc_cli_linuxx64_v11.1.tar.gz that you are trying to use does not come with include folder and it do not have any header files. So, you can not use it with any other open source driver like node-ibm_db or python-ibm_db. Its a wrong version of clidriver that get packaged with .net-core without header files. If ibm_data_server_driver_for_odbc_cli_linuxx64_v11.1.tar.gz would have include folder, then you should be able to use ibm_db with this clidriver and current license file. Hope, you'll understand the difference.

Please tell .net-core team to use clidriver from https://public.dhe.ibm.com/ibmdl/export/pub/software/data/db2/drivers/odbc_cli or keep include folder too with the clidriver that they ship with .net-core.

Unset IBM_DB_HOME using : export IBM_DB_HOME= npm install ibm_db get license file for DB2 V10.5fp6 copy 10.5 license file under ibm_db/installer/clidriver/license folder

Thanks.

kentonbmax commented 6 years ago

I will try that. I still fail to understand why ibm_db works on my windows 10 machine and not a linux container.

On Tue, Jun 5, 2018 at 1:39 AM Bimal Kumar Jha notifications@github.com wrote:

@kentonbmax https://github.com/kentonbmax Please note that the license file that you have is for db2 v11.1 and it will not work with ibm_db which uses DB2 v10.5. You need a db2 v10.5 db2connect license file for ibm_db.

The ibm_data_server_driver_for_odbc_cli_linuxx64_v11.1.tar.gz that you are trying to use does not come with include folder and it do not have any header files. So, you can not use it with any other open source driver like node-ibm_db or python-ibm_db. Its a wrong version of clidriver that get packaged with .net-core without header files. If ibm_data_server_driver_for_odbc_cli_linuxx64_v11.1.tar.gz would have include folder, then you should be able to use ibm_db with this clidriver and current license file. Hope, you'll understand the difference.

Please tell .net-core team to use clidriver from https://public.dhe.ibm.com/ibmdl/export/pub/software/data/db2/drivers/odbc_cli or keep include folder too with the clidriver that they ship with .net-core.

Unset IBM_DB_HOME using : export IBM_DB_HOME= npm install ibm_db get license file for DB2 V10.5fp6 copy 10.5 license file under ibm_db/installer/clidriver/license folder

Thanks.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/ibmdb/node-ibm_db/issues/217#issuecomment-394587872, or mute the thread https://github.com/notifications/unsubscribe-auth/APEoRFvqzFZXdoMVz21rIYWAY2W56Pqyks5t5hmEgaJpZM4LlHty .

kentonbmax commented 6 years ago

Can you please confirm what could lead this to work when running my application on windows 10 and not on linux? I do have Inavigator installed on windows 10. I wanted to add that I have not had to include a license file in the license folder when running on windows 10.

On Tue, Jun 5, 2018 at 8:14 AM Kenton Bocock kentonbo@gmail.com wrote:

I will try that. I still fail to understand why ibm_db works on my windows 10 machine and not a linux container.

On Tue, Jun 5, 2018 at 1:39 AM Bimal Kumar Jha notifications@github.com wrote:

@kentonbmax https://github.com/kentonbmax Please note that the license file that you have is for db2 v11.1 and it will not work with ibm_db which uses DB2 v10.5. You need a db2 v10.5 db2connect license file for ibm_db.

The ibm_data_server_driver_for_odbc_cli_linuxx64_v11.1.tar.gz that you are trying to use does not come with include folder and it do not have any header files. So, you can not use it with any other open source driver like node-ibm_db or python-ibm_db. Its a wrong version of clidriver that get packaged with .net-core without header files. If ibm_data_server_driver_for_odbc_cli_linuxx64_v11.1.tar.gz would have include folder, then you should be able to use ibm_db with this clidriver and current license file. Hope, you'll understand the difference.

Please tell .net-core team to use clidriver from https://public.dhe.ibm.com/ibmdl/export/pub/software/data/db2/drivers/odbc_cli or keep include folder too with the clidriver that they ship with .net-core.

Unset IBM_DB_HOME using : export IBM_DB_HOME= npm install ibm_db get license file for DB2 V10.5fp6 copy 10.5 license file under ibm_db/installer/clidriver/license folder

Thanks.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/ibmdb/node-ibm_db/issues/217#issuecomment-394587872, or mute the thread https://github.com/notifications/unsubscribe-auth/APEoRFvqzFZXdoMVz21rIYWAY2W56Pqyks5t5hmEgaJpZM4LlHty .

bimalkjha commented 6 years ago

@kentonbmax There seems some confusion. Lets talk about Windows first. You have not shared the complete output of db2level command and hence, we do not get complete info. The Install location and copy name is missing which gives more info about the product in use. See a sample db2level output that I get:

DB21085I  This instance or install (instance name, where applicable: "DB2")
uses "64" bits and DB2 code release "SQL11013" with level identifier
"0204010F".
Informational tokens are "DB2 v11.1.3030.148", "s1712010100",
"DYN1712010100WIN64", and Fix Pack "3".
Product is installed at "C:\PROGRA~1\IBM\SQLLIB" with DB2 Copy Name "DB2COPY1".

Seems, you are using DB2 v11.1.2 on Windows and you have license file for DB2 v11.1.2 too. Once you connect using the license file to i-Series, license check happens after 7 days only from that client. If you have never used license file on Windows to connect to i-Series, means server is configured to accept connection from this Windows client without client side license. Note that if server is configured to accept connection from your Windows system, then no db2connect license is required on your Windows system.
Now about linux: seems your i-Series server is not configured to accept connection from this linux system. So, you need to apply license at client side. The expectation is clear: Either activate license on server to accept connection from client or apply license file at client side. You may check with your sysadmin with the error that you get on linux and ask him to enable the connection from your linux client. Thanks.

tatemz commented 5 years ago

It's prolly a good idea to support musl Alpine environment... It's really tiny and awesome

VizorKit commented 4 years ago

@willfarrell what did you end up doing? I'm facing the same issue and would prefer to use the alpine image as it is the company standard.

bimalkjha commented 4 years ago

node-ibmdb works on Linux-x86-64 images of [amazonlinux](https://hub.docker.com//amazonlinux/), ubuntu and debian. It can not work on alpile as IBM Db2 Client Installers are not compiled on alpine linux using musl libc. So, none of the IBM Driver for Db2 will work on alpine linux. Thanks.

sliekens commented 4 years ago

+1 for adding Alpine support.

stijwilliams commented 4 years ago

Another +1 for alpine support.. If your goal is to build secure containers, ubuntu and debian fail to meet the requirement with all the bloat they include.

Install scan tools

See https://github.com/aquasecurity/trivy

RUN apk add curl \ && curl -sfL https://raw.githubusercontent.com/aquasecurity/trivy/master/contrib/install.sh | sh -s -- -b /usr/local/bin \ && trivy --no-progress --download-db-only

This will fail the docker image build if there are problems

RUN trivy filesystem --exit-code 1 --no-progress /

^^ This is secure, run the same on the other base images, and you will see all sorts of security vulnerabilities.