shawn-gang commented 1 year ago

Operating System Name: latest version of MacOS Monterey, M1 chip
db2level output from clidriver if in use: unsure
Target Db2 Server Version: unsure, some version of Db2 on Cloud that was working prior to this issue
ibm_db version: 3.1.0
For non-Windows, output of below commands: uname - Darwin uname -m - arm64
Value of below environment variables if set: IBM_DB_HOME: nothing/blank PATH: /Library/Frameworks/Python.framework/Versions/3.11/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/Library/Apple/usr/bin DYLD_LIBRARY_PATH: /usr/local/lib/gcc/8:

When database code is executed with the library, I get the same error as this user: https://github.com/ibmdb/node-ibm_db/issues/888.

This was a known issue to our team previously, and we were able to get around this error, even on newly setup laptops by running rm node_modules/ibm_db/installer/clidriver/lib/libstdc++.6.dylib after running the npm install commands. However after a fresh npm install on my computer, in which this library on the same version was working previously, the file node_modules/ibm_db/installer/clidriver/lib/libstdc++.6.dylib did not exist, and I ran into this error again.

bimalkjha commented 1 year ago

@shawn-gang Since, many users has to run rm node_modules/ibm_db/installer/clidriver/lib/libstdc++.6.dylib post installation to avoid error, we have stopped adding this dylib in clidriver itself. Now, to avoid macos issues, installation of gcc is pre-requirement. We already have libstdc++.6.dylib in gcc as its a gcc library. I hope you have installed gcc in your system. Please share output of below commands:

gcc -v
ls -l /usr/local/lib/gcc/8/libstd*
node_modules/ibm_db
npm install

Thanks.

shawn-gang commented 1 year ago

I am currently OOO, will be back in 2 weeks. I will provide an update then

shawn-gang commented 1 year ago

@shawn-gang Since, many users has to run rm node_modules/ibm_db/installer/clidriver/lib/libstdc++.6.dylib post installation to avoid error, we have stopped adding this dylib in clidriver itself. Now, to avoid macos issues, installation of gcc is pre-requirement. We already have libstdc++.6.dylib in gcc as its a gcc library. I hope you have installed gcc in your system. Please share output of below commands:
gcc -v
ls -l /usr/local/lib/gcc/8/libstd*
node_modules/ibm_db
npm install
Thanks.

hi, sorry for the delay. here are the outputs:

gcc -v

Apple clang version 14.0.0 (clang-1400.0.29.102)
Target: arm64-apple-darwin21.6.0
Thread model: posix
InstalledDir: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin

ls -l /usr/local/lib/gcc/8/libstd*

-rw-r--r--  1 sean  admin   2785544 30 Jan 10:51 /usr/local/lib/gcc/8/libstdc++.6.dylib
-r--r--r--  1 sean  admin  15134848 14 May  2021 /usr/local/lib/gcc/8/libstdc++.a
-r--r--r--  1 sean  admin      2432 30 Jan 10:51 /usr/local/lib/gcc/8/libstdc++.a-gdb.py
lrwxr-xr-x  1 sean  admin        17 14 May  2021 /usr/local/lib/gcc/8/libstdc++.dylib -> libstdc++.6.dylib
-r--r--r--  1 sean  admin  12028448 14 May  2021 /usr/local/lib/gcc/8/libstdc++fs.a

npm install ibm_db@latest


> ibm_db@3.1.0 install {parent directory}/node_modules/ibm_db
> node installer/driverInstall.js

platform = darwin, arch = x64, node.js version = v10.19.0
make version =GNU Make 3.81

****************************************
You are downloading a package which includes the Node.js module for IBM DB2/Informix.  The module is licensed under the Apache License 2.0. The package also includes IBM ODBC and CLI Driver from IBM, which is automatically downloaded as the node module is installed on your system/device. The license agreement to the IBM ODBC and CLI Driver is available in {parent directory}/node_modules/ibm_db/installer/clidriver. Check for additional dependencies, which may come with their own license agreement(s). Your use of the components of the package and dependencies constitutes your acceptance of their respective license agreements. If you do not accept the terms of any license agreement(s), then delete the relevant component(s) from your device.
****************************************

Downloading DB2 ODBC CLI Driver from https://public.dhe.ibm.com/ibmdl/export/pub/software/data/db2/drivers/odbc_cli/macos64_odbc_cli.tar.gz ...

100.00% | 29943201 bytes downloaded out of 29943201 bytes.

Downloading and extraction of DB2 ODBC CLI Driver completed successfully.

  CXX(target) Release/obj.target/odbc_bindings/src/odbc.o
  CXX(target) Release/obj.target/odbc_bindings/src/odbc_connection.o
  CXX(target) Release/obj.target/odbc_bindings/src/odbc_statement.o
  CXX(target) Release/obj.target/odbc_bindings/src/odbc_result.o
  SOLINK_MODULE(target) Release/odbc_bindings.node

ibm_db installed successfully.

+ ibm_db@3.1.0

I noticed that ibm_db@2.7.0 is also not working with the same error, and also the same absence of the file node_modules/ibm_db/installer/clidriver/lib/libstdc++.6.dylib

shawn-gang commented 1 year ago

ls -l output of the ibm_db directory in the node_modules folder

total 296
-rw-r--r--    1 sean  staff  68426 26 Oct  1985 APIDocumentation.md
-rw-r--r--    1 sean  staff  27311 26 Oct  1985 CHANGES.md
-rw-r--r--    1 sean  staff  12134 26 Oct  1985 INSTALL.md
-rw-r--r--    1 sean  staff   1149 26 Oct  1985 LICENSE
-rw-r--r--    1 sean  staff  23997 26 Oct  1985 README.md
-rw-r--r--    1 sean  staff     26 26 Oct  1985 _config.yml
-rw-r--r--    1 sean  staff   3384 26 Oct  1985 binding.gyp
drwxr-xr-x    8 sean  staff    256  3 Apr 11:55 build
drwxr-xr-x    3 sean  staff     96  3 Apr 11:55 contributing
drwxr-xr-x    5 sean  staff    160  3 Apr 11:55 examples
drwxr-xr-x    8 sean  staff    256  3 Apr 11:55 installer
drwxr-xr-x    5 sean  staff    160  3 Apr 11:55 lib
drwxr-xr-x    5 sean  staff    160  3 Apr 11:55 node_modules
-rw-r--r--    1 sean  staff   1952  3 Apr 11:55 package.json
drwxr-xr-x   12 sean  staff    384  3 Apr 11:55 src
drwxr-xr-x  123 sean  staff   3936  3 Apr 11:55 test

I noticed there is no directory ibm_db/clidriver so previous fixes for this issue don't apply

shawn-gang commented 1 year ago

I should also note I am on a corporate laptop so actions like sudo cp /usr/local/lib/gcc/8/libstdc++.6.dylib /usr/lib/libstdc++.6.0.9.dylib don't work as they are protected directories that shouldn't be unlocked to make manual changes

bimalkjha commented 1 year ago

@shawn-gang I noticed there is no directory ibm_db/clidriver so previous fixes for this issue don't apply

clidriver should be available as ibm_db/installer/clidriver. Please check and confirm. Also, run this command from terminal and verify the issue: export DYLD_LIBRARY_PATH=/usr/local/lib/gcc/8:....../node_modules/ibm_db/installer/clidriver/lib If still not working then cp /usr/local/lib/gcc/8/libstdc++.6.dylib ..../node_modules/ibm_db/installer/clidriver/lib and try. Thanks.

shawn-gang commented 1 year ago

hi @bimalkjha, the above fixes don't work I also tried deleting the file node_modules/ibm_db/installer/clidriver/lib/libstdc++.6.dylib which did not work as well

bimalkjha commented 1 year ago

@shawn-gang I noticed output of your gcc -v command. It is showing Target: arm64-apple-darwin21.6.0. So the gcc library is incompatible with clidriver which is x64. We need x64 version of both nodejs and gcc on arm64 system for ibm_db. The gcc -v command should show Target: x86_64-apple-darwin21 in output. Please install gcc8 using homebrew by following below commands, then check gcc -v output and verify your program.

brew tap homebrew/versions
brew install homebrew/versions/gcc8

If it do not work or throw error, then

brew tap homebrew/homebrew-core
brew update
brew search gcc@8
brew install gcc@8
ls /usr/local/lib/gcc

Now set DYLD_LIBRARY_PATH env var pointing to gcc8 If still issue not resolved, then please share output of below commands:

ls /usr/local/lib/gcc
gcc -v
cd clidriver/lib
otool -L libdb2.dylib

Thanks.

bimalkjha commented 1 year ago

Steps to install x86_64 gcc@8 on arm64/M1 architecture

Warning: If you use the ARM version of homebrew (as recommended for M1 architectures) you will get the following error message:

$ brew install gcc-8
Error: Cannot install in Homebrew on ARM processor in Intel default prefix (/usr/local)!
Please create a new installation in /opt/homebrew using one of the
"Alternative Installs" from:
  https://docs.brew.sh/Installation
You can migrate your previously installed formula list with:
  brew bundle dump

Install gcc-8 using homebrew (note: the x86_64 version of homebrew is needed for this, not the recommended ARM based homebrew). The clearest instructions on how to install and use the x86_64 version of homebrew is by following below steps:

My arm64/M1 brew is installed here:
```
$ which brew
/opt/homebrew/bin/brew
```
Step 1. Install x86_64 brew under /usr/local/bin/brew arch -x86_64 /bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install.sh)"
Step 2. Create an alias for the x86_64 brew I added this to my ~/.bashrc as below:
```
# brew hack for x86_64
alias brew64='arch -x86_64 /usr/local/bin/brew'
```
Then install gcc-8 using the x86_64 homebrew:
```
brew64 install gcc@8
```
Thanks.

huineng commented 1 year ago

thank you for this, i was still deleting that file :)

shawn-gang commented 1 year ago

Did a fresh uninstall of brew on my computer before beginning, followed the uninstall script here https://github.com/homebrew/install#uninstall-homebrew:

% brew
zsh: command not found: brew
% brew64
arch: /usr/local/bin/brew isn't executable
% gcc -v
Apple clang version 14.0.0 (clang-1400.0.29.102)
Target: arm64-apple-darwin21.6.0
Thread model: posix
InstalledDir: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin

As noted above, it seems that gcc is included in xcode, as gcc -v outputs something even after brew is uninstalled, as well as if gcc@8 is uninstalled through brew.

Installed x86_64 brew, and added alias to shell initialization file

arch -x86_64 /bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install.sh)"
...
nano ~/.zshrc
...

At this point when I run gcc -v, I still get arm64, but when I start a new shell, run this command: arch -x86_64 zsh, I get x84_64 when I run the command. I opened a fresh terminal, put in arch -x86_64 zsh and then tried to run my program, but I still received the malloc error. I receive the malloc error without running that x86_64 command as well.

Output requested:

% ls /usr/local/lib/gcc
8

% gcc -v
Apple clang version 14.0.0 (clang-1400.0.29.102)
Target: arm64-apple-darwin21.6.0

% arch -x86_64 zsh     
% gcc -v
Apple clang version 14.0.0 (clang-1400.0.29.102)
Target: x86_64-apple-darwin21.6.0

% otool -L libdb2.dylib
libdb2.dylib:
    libdb2.dylib (compatibility version 0.0.0, current version 0.0.0)
    /System/Library/Frameworks/CoreFoundation.framework/Versions/A/CoreFoundation (compatibility version 150.0.0, current version 1677.104.0)
    /System/Library/Frameworks/IOKit.framework/Versions/A/IOKit (compatibility version 1.0.0, current version 275.0.0)
    /usr/lib/libresolv.9.dylib (compatibility version 1.0.0, current version 1.0.0)
    /usr/local/lib/gcc/8/libstdc++.6.dylib (compatibility version 7.0.0, current version 7.25.0)
    /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1281.100.1)
    /usr/local/lib/gcc/8/libgcc_s.1.dylib (compatibility version 1.0.0, current version 1.0.0)

Here is an output of my ~/.zshrc file:

export ARCHFLAGS="-arch x86_64"
export DYLD_LIBRARY_PATH=/usr/local/lib/gcc/8:{ABSOLUTE PATH TO MY APPLICATION THAT USES NODE_IBM_DB}/node_modules/ibm_db/installer/clidriver/lib:$DYLD_LIBRARY_PATH

# brew hack for x86_64
alias brew64='arch -x86_64 /usr/local/bin/brew'

shawn-gang commented 1 year ago

If it's helpful at all, to setup the ibm_db library in Python to work on the M1 macbooks, our team follows a similar workflow of:

running arch -x86_64 zsh
installing brew
installing Python and gcc@8 through brew

We receive the same malloc error if we don't follow these (for Python)

bimalkjha commented 1 year ago

@shawn-gang Please confirm that by following above steps, your malloc error issue got resolved or not. Then I'll add using terminal arch -x86_64 zsh in our steps too. Thanks.

shawn-gang commented 1 year ago

@bimalkjha the malloc issue is still not resolved. Running that command in the terminal before running the program still results in the malloc issue

bimalkjha commented 1 year ago

@shawn-gang The malloc error comes due to incompatible libgcc_s.1.dylib library or this library is not found by the clidriver. If gcc8 is in path and otool -L libdb2.dylib is able to find all libraries, then it should work. Probably I need to look into your system and check. Is it possible to have a webex call for troubleshooting? If yes, please let me know your preferred time and I can share webex meeting invitation. Or you can also schedule any time tomorrow between 11AM to 10PM IST. My emailid is bjha@rocketsoftware.com. Thanks.

shawn-gang commented 1 year ago

@bimalkjha started an email

shawn-gang commented 1 year ago

The issue has been resolved. A summary of steps taken to reach resolution

arch -x86_64 /bin/bash  -- points gcc to the x86_64 version

cp /usr/local/lib/gcc/8/libstdc++.6.dylib {path_to_application}/node_modules/ibm_db/installer/clidriver/lib

In the ~/.zshrc file:

export ARCHFLAGS="-arch x86_64"
export DYLD_LIBRARY_PATH=/usr/local/lib/gcc/8:{ABSOLUTE PATH TO MY APPLICATION THAT USES NODE_IBM_DB}/node_modules/ibm_db/installer/clidriver/lib:$DYLD_LIBRARY_PATH

Doing a fresh reinstall of ibm_db may have helped too, as opposed to doing rm -rf node_modules

npm uninstall ibm_db

npm install ibm_db

thanks again @bimalkjha

ibmdb / node-ibm_db

malloc error when using library #911

Steps to install x86_64 gcc@8 on arm64/M1 architecture