search

ibmdb / python-ibmdb

Automatically exported from code.google.com/p/ibm-db
Apache License 2.0
308 stars 192 forks source link

malloc error on Monetrey #789

Closed mjdavislumen closed 2 years ago

mjdavislumen commented 2 years ago

Reopening #786 I'd like to reopen the issue that is mentioned in #786 wrt the malloc problem. I am still not able to use ibm_db in python. I have carefully documented what I have done and their results below os= Monterrey 12.5.1 db2 package: dsdriver (NOT CLI) "IBM Data Server Driver Package (DS Driver) V11.5.7" gcc: gcc@8--8.5.0.monterey python: Python 3.9.13

STEP 1 Install dsdriver After installation

$ cd /Applications/dsdriver/bin
$ ./db2level
DB21085I  This instance or install (instance name, where applicable: "*") uses
"64" bits and DB2 code release "SQL11057" with level identifier "0608010F".
Informational tokens are "DB2 v11.5.7.0", "s2111221000", "DYN2111221000MAC",
and Fix Pack "0".
Product is installed at "/Applications/dsdriver".

I have confirmed this can connect to a remote database (it's a DB2 Warehouse on IBM Cloud)

STEP 2 - Fix the libraries in dsdriver

$ cd /Applications/dsdriver/lib/
$ install_name_tool -change /usr/local/opt/gcc@8/lib/gcc/8/libstdc++.6.dylib /usr/local/lib/gcc/8/libstdc++.6.dylib  libdb2.dylib
$ export DYLD_LIBRARY_PATH=/Applications/dsdriver/lib:/Applications/dsdriver/bin:/usr/local/lib/gcc/8:$DYLD_LIBRARY_PATH

STEP 3 - install ibm_db

$ brew install gcc@8
$ export IBM_DB_HOME=/Applications/dsdriver
$ pip uninstall ibm_db
# I only have one cached wheel so
$ pip cache purge
$ pip install ibm_db
Collecting ibm_db
  Downloading ibm_db-3.1.3.tar.gz (1.4 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.4/1.4 MB 11.1 MB/s eta 0:00:00
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Installing backend dependencies ... done
  Preparing metadata (pyproject.toml) ... done
Building wheels for collected packages: ibm_db
  Building wheel for ibm_db (pyproject.toml) ... done
  Created wheel for ibm_db: filename=ibm_db-3.1.3-cp39-cp39-macosx_12_0_x86_64.whl size=375443 sha256=b95f9ed471bfa19840caa5e6f2b879d8a59eadf51eef8c8a4756542c297e45e6
  Stored in directory: /Users/matt/Library/Caches/pip/wheels/3d/6e/19/64e70ce3dde2ccda5c9b35bd6a313a39e46f6af0222c75cc5f
Successfully built ibm_db
Installing collected packages: ibm_db
Successfully installed ibm_db-3.1.3

STEP 4 - Test The file dbconnect.py contains

import ibm_db
dsn='DATABASE=bludb;HOSTNAME=b1bc1829-6f45-4cd4-bef4-10cf081900bf.c1ogj3sd0tgtu0lqde00.databases.appdomain.cloud;PORT=32304;PROTOCOL=TCPIP;UID=<USER_NAME>;PWD=<PASSWORD>;SECURITY=ssl;'
ibm_db.connect(dsn,"","")

Here is the current run of the python code

$ python dbconnect.py
python(64922,0x10a51e600) malloc: *** error for object 0x7ffb53df0080: pointer being freed was not allocated
python(64922,0x10a51e600) malloc: *** set a breakpoint in malloc_error_break to debug
zsh: abort      python dbconnect.py

otool and file output

$ otool -L /Applications/dsdriver/lib/libdb2.dylib
/Applications/dsdriver/lib/libdb2.dylib:
    libdb2.dylib (compatibility version 0.0.0, current version 0.0.0)
    /System/Library/Frameworks/CoreFoundation.framework/Versions/A/CoreFoundation (compatibility version 150.0.0, current version 1677.104.0)
    /System/Library/Frameworks/IOKit.framework/Versions/A/IOKit (compatibility version 1.0.0, current version 275.0.0)
    /usr/lib/libresolv.9.dylib (compatibility version 1.0.0, current version 1.0.0)
    /usr/local/lib/gcc/8/libstdc++.6.dylib (compatibility version 7.0.0, current version 7.25.0)
    /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1281.100.1)
    /usr/local/lib/gcc/8/libgcc_s.1.dylib (compatibility version 1.0.0, current version 1.0.0)
$ file /usr/local/lib/gcc/8/libstdc++.6.dylib
/usr/local/lib/gcc/8/libstdc++.6.dylib: Mach-O 64-bit dynamically linked shared library x86_64
$ file /usr/local/lib/gcc/8/libgcc_s.1.dylib
/usr/local/lib/gcc/8/libgcc_s.1.dylib: Mach-O universal binary with 1 architecture: [x86_64:Mach-O 64-bit dynamically linked shared library x86_64Mach-O 64-bit dynamically linked shared library x86_64]
/usr/local/lib/gcc/8/libgcc_s.1.dylib (for architecture x86_64):    Mach-O 64-bit dynamically linked shared library x86_64
$ otool -L /Users/matt/.pyenv/versions/3.9.13/lib/python3.9/site-packages/ibm_db.cpython-39-darwin.so
/Users/matt/.pyenv/versions/3.9.13/lib/python3.9/site-packages/ibm_db.cpython-39-darwin.so:
    /Applications/dsdriver/lib/libdb2.dylib (compatibility version 0.0.0, current version 0.0.0)
    /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1311.100.3)
$ gcc -v
Apple clang version 13.1.6 (clang-1316.0.21.2.5)
Target: x86_64-apple-darwin21.6.0
Thread model: posix
InstalledDir: /Library/Developer/CommandLineTools/usr/bin

Any thoughts?

bimalkjha commented 2 years ago

@mjdavislumen Please share output of xcodebuild -version command. If it fails, please install XCode from App Store and verify your python program. Thanks.

bimalkjha commented 2 years ago

Also, if possible, copy /usr/local/lib/gcc/8/libstdc++.6.dylib as /usr/lib/libstdc++.6.dylib. Thanks.

mjdavislumen commented 2 years ago

Bit of a long answer. When I first ran the command I got an error

$ xcodebuild -version
xcode-select: error: tool 'xcodebuild' requires Xcode, but active developer directory '/Library/Developer/CommandLineTools' is a command line tools instance

$ xcode-select -p
/Library/Developer/CommandLineTools

This seemed really weird as I had installed Xcode, so decided to uninstall Xcode and reinstall it. HOWEVER, as I was about to do this I got informed that a Monterey update was available, and without thinking I updated. So I am now at Version Monterey 12.6. Probably should of held off doing this until I had this ibm_db problem sorted, but my fingers got ahead of my brain. OOOPS!!

Anyway after the update I ran the same commands above and l got the same errors, so I decided to uninstall XCode and the command line tools. 1) I uninstalled Xcode completely by deleting the package from /Applications 2) I deleted the command line tools using sudo rm -rf /Library/Developer/CommandLineTools 3) I rebooted 4) I reinstalled XCode from the App Store 5) I ran the xcodebuild -version and xcode-select -p and got the same result as above. 6) Then I re-read some articles about reinstalling Xcode and they indicated you have to START XCode to complete the installation. So I did. I actually hunted through the "Preferences" and found an option under "Locations" to select Command Line Tools. So I selected 'Xcode 14.0 (14A309)'

  1. Now I ran the above commands with different results.
$xcodebuild -version
Xcode 14.0
Build version 14A309

$ xcode-select -p
/Applications/Xcode.app/Contents/Developer

I then performed a full uninstall of ibm_db, purged the cache, and rebuilt the code, i.e.

$ pip uninstall ibm_db
Found existing installation: ibm-db 3.1.3
Uninstalling ibm-db-3.1.3:
  Would remove:
    /Users/matt/.pyenv/versions/3.9.13/lib/python3.9/site-packages/Users/matt/.pyenv/versions/3.9.13/lib/python3.9/site-packages/CHANGES.md
    /Users/matt/.pyenv/versions/3.9.13/lib/python3.9/site-packages/Users/matt/.pyenv/versions/3.9.13/lib/python3.9/site-packages/LICENSE
    /Users/matt/.pyenv/versions/3.9.13/lib/python3.9/site-packages/Users/matt/.pyenv/versions/3.9.13/lib/python3.9/site-packages/README.md
    /Users/matt/.pyenv/versions/3.9.13/lib/python3.9/site-packages/Users/matt/.pyenv/versions/3.9.13/lib/python3.9/site-packages/config.py.sample
    /Users/matt/.pyenv/versions/3.9.13/lib/python3.9/site-packages/certs/*
    /Users/matt/.pyenv/versions/3.9.13/lib/python3.9/site-packages/ibm_db-3.1.3.dist-info/*
    /Users/matt/.pyenv/versions/3.9.13/lib/python3.9/site-packages/ibm_db.cpython-39-darwin.so
    /Users/matt/.pyenv/versions/3.9.13/lib/python3.9/site-packages/ibm_db_dbi.py
    /Users/matt/.pyenv/versions/3.9.13/lib/python3.9/site-packages/ibm_db_dlls/*
    /Users/matt/.pyenv/versions/3.9.13/lib/python3.9/site-packages/ibm_db_tests/*
    /Users/matt/.pyenv/versions/3.9.13/lib/python3.9/site-packages/testfunctions.py
    /Users/matt/.pyenv/versions/3.9.13/lib/python3.9/site-packages/tests.py
  Would not remove (might be manually added):
    /Users/matt/.pyenv/versions/3.9.13/lib/python3.9/site-packages/ibm_db_dlls/.DS_Store
Proceed (Y/n)? y
  Successfully uninstalled ibm-db-3.1.3
$ pip cache purge
Files removed: 7
$ echo $DYLD_LIBRARY_PATH
/Applications/dsdriver/lib:/Applications/dsdriver/bin:/usr/local/lib/gcc/8
$ echo $IBM_DB_HOME
/Applications/dsdriver
$ pip install ibm_db
Collecting ibm_db
  Downloading ibm_db-3.1.3.tar.gz (1.4 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.4/1.4 MB 9.6 MB/s eta 0:00:00
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Installing backend dependencies ... done
  Preparing metadata (pyproject.toml) ... done
Building wheels for collected packages: ibm_db
  Building wheel for ibm_db (pyproject.toml) ... done
  Created wheel for ibm_db: filename=ibm_db-3.1.3-cp39-cp39-macosx_12_0_x86_64.whl size=375347 sha256=5fee0e4954f35727d3e9d73ce0ad6e666dc6bb9cadaa4134d86c9682923eca42
  Stored in directory: /Users/matt/Library/Caches/pip/wheels/3d/6e/19/64e70ce3dde2ccda5c9b35bd6a313a39e46f6af0222c75cc5f
Successfully built ibm_db
Installing collected packages: ibm_db
Successfully installed ibm_db-3.1.3
$ python dbconnect.py
python(13060,0x113279600) malloc: *** error for object 0x7ffb5850a440: pointer being freed was not allocated
python(13060,0x113279600) malloc: *** set a breakpoint in malloc_error_break to debug
zsh: abort      python dbconnect.py

so still no luck.

What's is your next thought?

And P.S. I really appreciate you helping me with this, as I am sure it is driving us both crazy :-)

mjdavislumen commented 2 years ago 
$ ls -al /usr/lib/libstdc++.6.dylib
lrwxr-xr-x  1 root  wheel  21 Aug 24 02:59 /usr/lib/libstdc++.6.dylib -> libstdc++.6.0.9.dylib
$ sudo rm  /usr/lib/libstdc++.6.dylib
Password: 
rm: /usr/lib/libstdc++.6.dylib: Operation not permitted

That linked file does not exist and I can not delete it without disabling Mac's System Integrity Protection (SIP), which right now I am unwilling to do.

Any other ideas?

bimalkjha commented 2 years ago

@mjdavislumen I would suggest to copy /usr/local/lib/gcc/8/libstdc++.6.dylib as /usr/lib/libstdc++.6.0.9.dylib as this file does not exists. Thanks.

bimalkjha commented 2 years ago

@mjdavislumen Please update if you are facing this issue after copying /usr/local/lib/gcc/8/libstdc++.6.dylib as /usr/lib/libstdc++.6.0.9.dylib? Thanks.