search

ContinuumIO / anaconda-issues

Anaconda issue tracking
648 stars 221 forks source link

Anaconda Python + pyodbc does not work #1639

Closed kalefranz closed 7 years ago

kalefranz commented 7 years ago

From @meet-bhagdev on April 26, 2017 2:41

Anaconda Python is not able to run a python script successfully that works with system python.

To repro:

$pip install pyodbc
$sudo su 
$curl https://packages.microsoft.com/keys/microsoft.asc | apt-key add -
$curl https://packages.microsoft.com/config/ubuntu/16.04/prod.list > /etc/apt/sources.list.d/mssql-release.list
$exit
$sudo apt-get update
$sudo ACCEPT_EULA=Y apt-get install msodbcsql=13.1.6.0-1 mssql-tools=14.0.3.0-1 unixodbc-dev
$pip install pyodbc
$vim test.py

Paste the following in test.py

import pyodbc
server = 'localhost'
database = 'SampleDB'
username = 'sa'
password = 'your_password'
cnxn = pyodbc.connect('DRIVER={ODBC Driver 13 for SQL Server};SERVER='+server+';PORT=1433;DATABASE='+database+';UID='+username+';PWD='+ password)
cursor = cnxn.cursor()
print ('Reading data from table')
tsql = "SELECT @@VERSION;"
with cursor.execute(tsql):
    row = cursor.fetchone()
    while row:
        print (str(row[0]))
        row = cursor.fetchone()

$ python test.py -- Works! 

$cd /usr/bin/anaconda/bin/
$./pip install pyodbc
$./python test.py

This gives an error: pyodbc.Error: ('01000', "[01000] [unixODBC][Driver Manager]Can't open lib '/opt/microsoft/msodbcsql/lib64/libmsodbcsql-13.1.so.6.0' : file not found (0) (SQLDriverConnect)")

I think it is something about permissions (anaconda python not being able to read from certain folders)

Some other issues on StackOverflow: http://stackoverflow.com/questions/42301519/pyodbc-cannot-find-unixodbc-library-when-using-anaconda-python

Copied from original issue: conda/conda#5150

nehaljwani commented 7 years ago

@meet-bhagdev Does the following help your issue:

pip uninstall pyodbc
conda install pyodbc
meet-bhagdev commented 7 years ago

I tried this. This doesn't help as conda install pyodbc for installs unixODBC 2.3.4. The ODBC Driver being used is only compatible with unixODBC 2.3.1. Were you able to repro this issue?

nehaljwani commented 7 years ago

@meet-bhagdev The 2.3.1 version of unixodbc is also available in the defaults channel:

(root) bash-4.1# conda install pyodbc unixodbc=2.3.1

By the way, I was not able to reproduce the error scenario using your steps (I used a docker container for ubuntu:16.04 and microsoft/mssql-server-linux), but was able to trigger the exact error when I did sudo mv /opt/microsoft/ /opt/microsoft_, irrespective of whether I used conda's python or system's python; so I am failing to see how this is related to conda.

If you are still able to reproduce this, please type:

strace -fo /tmp/pyodbc.debug -s 100 /usr/bin/anaconda/bin/python /path/to/test.py

and then attach the debug file /tmp/pyodbc.debug with a comment here.

meet-bhagdev commented 7 years ago

@nehaljwani awesome :) let me try this.

xusun commented 7 years ago

tried the process, but still got the same error. Here's the pyodbc.debug file. pyodbc.debug.txt

nehaljwani commented 7 years ago

@xusun Thank you for the file.

From the output it seems as if your file test.py has the library location hard coded?

cnxn = pyodbc.connect('DRIVER={/home/libmsodbcsql-13.1.so.4.0};SERVER=...

Instead of 

cnxn = pyodbc.connect('DRIVER={ODBC Driver 13 for SQL Server};SERVER=...

And since /home/libmsodbcsql-13.1.so.4.0 is not present, it fails (which is expected):

open("/home/libmsodbcsql-13.1.so.4.0", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)

If the same test.py works for you outside anaconda, would it be also possible for you to share the output for:

strace -fo /tmp/pyodbc2.debug -s 100 /path/to/system/python /path/to/test.py
nehaljwani commented 7 years ago

Also, with anaconda's python, you were using /usr/bin/anaconda/bin/test.py.

lstat("/usr/bin/anaconda/bin/test.py", {st_mode=S_IFREG|0644, st_size=509, ...}) = 0

Is this the same file you were using for the working, system python?

xusun commented 7 years ago

this causes system python to complain can't find module pyodbc

pip uninstall pyodbc
conda install pyodbc

so, I had to pip install pyodbc

Now, I made sure the two use the same test.py and here are the debug logs. pyodbc2.debug.txt pyodbc.debug.txt

nehaljwani commented 7 years ago

In pyodbc.debug.txt, we have:

7603  open("/etc/odbcinst.ini", O_RDONLY) = 3
...
7603  open("/opt/microsoft/msodbcsql/lib64/libmsodbcsql-13.1.so.6.0", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)

In pyodbc2.debug.txt, we have:

7573  open("/usr/local/etc/odbcinst.ini", O_RDONLY) = 3
...
7573  open("/opt/microsoft/msodbcsql/lib64/libmsodbcsql-13.1.so.4.0", O_RDONLY|O_CLOEXEC) = 3

It is evident that pyodbc/unixodbc (installed via pip) is picking up driver configuration from /usr/local/etc/odbcinst.ini and the pyodbc/unixodbc (installed via conda) is picking up driver configuration from /etc/odbcinst.ini.

@xusun Could you please confirm that the contents of these two ini files differ in the location of the shared library? (Driver=/opt/micr...)

xusun commented 7 years ago

Yes. That's a mismatch.
So, I manually updated the /etc/odbcinst.ini and changed it to "/opt/microsoft/msodbcsql/lib64/libmsodbcsql-13.1.so.6.0".
Here's the new trace.
pyodbc3.debug.txt

nehaljwani commented 7 years ago

@xusun I think you mean, you changed it to /opt/microsoft/msodbcsql/lib64/libmsodbcsql-13.1.so.4.0 .

According to the output, it did find the file this time:

4568  open("/opt/microsoft/msodbcsql/lib64/libmsodbcsql-13.1.so.4.0", O_RDONLY|O_CLOEXEC) = 3

Could you please share the output for:

cd /usr/bin/anaconda/bin/
ldd /opt/microsoft/msodbcsql/lib64/libmsodbcsql-13.1.so.4.0

Does the same error occur when you create a new environment:

conda create -yn odbctest pyodbc unixodbc=2.3.1
source activate odbctest
python /path/to/test.py

Even with msodbcsql=13.1.4.0-1, I am not able to reproduce this error :'(

xusun commented 7 years ago

Yes. That's what I meant, to ...4.0

The output from ldd: linux-vdso.so.1 => (0x00007ffd35276000) libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007ff9a4894000) libodbcinst.so.2 => /usr/local/lib/libodbcinst.so.2 (0x00007ff9a4682000) libuuid.so.1 => /lib/x86_64-linux-gnu/libuuid.so.1 (0x00007ff9a447c000) libcurl.so.4 => /usr/lib/x86_64-linux-gnu/libcurl.so.4 (0x00007ff9a420d000) libgssapi_krb5.so.2 => /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2 (0x00007ff9a3fc3000) libkrb5.so.3 => /usr/lib/x86_64-linux-gnu/libkrb5.so.3 (0x00007ff9a3cf0000) libcrypto.so.1.0.0 => /opt/omi/lib/libcrypto.so.1.0.0 (0x00007ff9a38ac000) libssl.so.1.0.0 => /opt/omi/lib/libssl.so.1.0.0 (0x00007ff9a3643000) libstdc++.so.6 => /usr/lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007ff9a32c0000) libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007ff9a2fb7000) libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007ff9a2da1000) libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007ff9a2b83000) libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007ff9a27ba000) /lib64/ld-linux-x86-64.so.2 (0x0000557afa39a000) libltdl.so.7 => /usr/lib/x86_64-linux-gnu/libltdl.so.7 (0x00007ff9a25b0000) libidn.so.11 => /usr/lib/x86_64-linux-gnu/libidn.so.11 (0x00007ff9a237c000) librtmp.so.1 => /usr/lib/x86_64-linux-gnu/librtmp.so.1 (0x00007ff9a2160000) liblber-2.4.so.2 => /usr/lib/x86_64-linux-gnu/liblber-2.4.so.2 (0x00007ff9a1f51000) libldap_r-2.4.so.2 => /usr/lib/x86_64-linux-gnu/libldap_r-2.4.so.2 (0x00007ff9a1cff000) libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1 (0x00007ff9a1ae5000) libk5crypto.so.3 => /usr/lib/x86_64-linux-gnu/libk5crypto.so.3 (0x00007ff9a18b6000) libcom_err.so.2 => /lib/x86_64-linux-gnu/libcom_err.so.2 (0x00007ff9a16b1000) libkrb5support.so.0 => /usr/lib/x86_64-linux-gnu/libkrb5support.so.0 (0x00007ff9a14a6000) libkeyutils.so.1 => /lib/x86_64-linux-gnu/libkeyutils.so.1 (0x00007ff9a12a2000) libresolv.so.2 => /lib/x86_64-linux-gnu/libresolv.so.2 (0x00007ff9a1086000) libgnutls.so.30 => /usr/lib/x86_64-linux-gnu/libgnutls.so.30 (0x00007ff9a0d56000) libhogweed.so.4 => /usr/lib/x86_64-linux-gnu/libhogweed.so.4 (0x00007ff9a0b22000) libnettle.so.6 => /usr/lib/x86_64-linux-gnu/libnettle.so.6 (0x00007ff9a08ec000) libgmp.so.10 => /usr/lib/x86_64-linux-gnu/libgmp.so.10 (0x00007ff9a066c000) libsasl2.so.2 => /usr/lib/x86_64-linux-gnu/libsasl2.so.2 (0x00007ff9a0450000) libgssapi.so.3 => /usr/lib/x86_64-linux-gnu/libgssapi.so.3 (0x00007ff9a020f000) libp11-kit.so.0 => /usr/lib/x86_64-linux-gnu/libp11-kit.so.0 (0x00007ff99ffaa000) libtasn1.so.6 => /usr/lib/x86_64-linux-gnu/libtasn1.so.6 (0x00007ff99fd97000) libheimntlm.so.0 => /usr/lib/x86_64-linux-gnu/libheimntlm.so.0 (0x00007ff99fb8d000) libkrb5.so.26 => /usr/lib/x86_64-linux-gnu/libkrb5.so.26 (0x00007ff99f903000) libasn1.so.8 => /usr/lib/x86_64-linux-gnu/libasn1.so.8 (0x00007ff99f661000) libhcrypto.so.4 => /usr/lib/x86_64-linux-gnu/libhcrypto.so.4 (0x00007ff99f42e000) libroken.so.18 => /usr/lib/x86_64-linux-gnu/libroken.so.18 (0x00007ff99f217000) libffi.so.6 => /usr/lib/x86_64-linux-gnu/libffi.so.6 (0x00007ff99f00f000) libwind.so.0 => /usr/lib/x86_64-linux-gnu/libwind.so.0 (0x00007ff99ede6000) libheimbase.so.1 => /usr/lib/x86_64-linux-gnu/libheimbase.so.1 (0x00007ff99ebd6000) libhx509.so.5 => /usr/lib/x86_64-linux-gnu/libhx509.so.5 (0x00007ff99e98b000) libsqlite3.so.0 => /usr/lib/x86_64-linux-gnu/libsqlite3.so.0 (0x00007ff99e6b6000) libcrypt.so.1 => /lib/x86_64-linux-gnu/libcrypt.so.1 (0x00007ff99e47d000)

I followed the steps for new environment and got the same error. pyodbc.Error: ('01000', "[01000] [unixODBC][Driver Manager]Can't open lib '/opt/microsoft/msodbcsql/lib64/libmsodbcsql-13.1.so.4.0' : file not found (0) (SQLDriverConnect)")

nehaljwani commented 7 years ago

@xusun According to pyodbc3.debug.txt , it seems as if it tries to load /opt/microsoft/msodbcsql/lib64/libmsodbcsql-13.1.so.4.0 two times, but fails for some unknown reason. Could you please share the debug files generated using the commands:

LD_DEBUG=all LD_DEBUG_OUTPUT=/tmp/debug_nw /usr/bin/anaconda/bin/python /home/sshadmin/test.py

and

LD_DEBUG=all LD_DEBUG_OUTPUT=/tmp/debug_w /usr/bin/python /home/sshadmin/test.py
xusun commented 7 years ago

debug_w.60065.txt debug_nw.60049.txt

nehaljwani commented 7 years ago

Aha! Finally!! I was able to reproduce your problem. 

(test) root@75eb43ff7f79:~$ python test.py
Traceback (most recent call last):
  File "test.py", line 6, in <module>
    cnxn = pyodbc.connect('DRIVER={ODBC Driver 13 for SQL Server};SERVER='+server+';PORT=1433;UID='+username+';PWD='+ password)
pyodbc.Error: ('01000', "[01000] [unixODBC][Driver Manager]Can't open lib '/opt/microsoft/msodbcsql/lib64/libmsodbcsql-13.1.so.4.0' : file not found (0) (SQLDriverConnect)")

(test) root@75eb43ff7f79:~$ ls -l /opt/microsoft/msodbcsql/lib64/libmsodbcsql-13.1.so.4.0
-rwxr-xr-x. 1 root root 15530176 Jan 24 23:42 /opt/microsoft/msodbcsql/lib64/libmsodbcsql-13.1.so.4.0

The last debug output helped :)

This is the culprit:

60049:     /usr/bin/anaconda/lib/python2.7/site-packages/../../libstdc++.so.6: error: version lookup error: version `GLIBCXX_3.4.20' not found (required by /opt/microsoft/msodbcsql/lib64/libmsodbcsql-13.1.so.4.0) (fatal)

In the pip case, pyodbc.so links to system's libstdc++.so. In conda's case, pyodbc.so tries to find libstdc++.so.6 in it's vicinity, due to relative RPATH:

(test) root@75eb43ff7f79:~$ chrpath -l $CONDA_PREFIX/lib/python2.7/site-packages/pyodbc.so
/conda/envs/test/lib/python2.7/site-packages/pyodbc.so: RPATH=$ORIGIN/../..

Since you have libgcc 4.8* in your conda environment, and it satisfies pyodbc.so's minimum requirements, it loads that one, instead of the system one.

However, the requirements of libmsodbcsql.so are pretty high:

(test) root@75eb43ff7f79:~$ readelf --version-info /opt/microsoft/msodbcsql/lib64/libmsodbcsql-13.1.so.4.0 | grep -Po '(?<=GLIBCXX_)([\d.]*)' | sort -Vr | head -1
3.4.21

And the highest version supported by libstdc++.so from libgcc (provided by conda):

(test) root@75eb43ff7f79:~$ readelf --version-info /conda/envs/test/lib/libstdc++.so | grep -Po '(?<=GLIBCXX_)([\d.]*)' | sort -Vr | head -1
3.4.19

And the highest version supported by the system's libstdc++.so:

(test) root@75eb43ff7f79:~$ readelf --version-info  /usr/lib/x86_64-linux-gnu/libstdc++.so.6 | grep -Po '(?<=GLIBCXX_)([\d.]*)' | sort -Vr | head -1
3.4.21

The official page also suggests that the minimum version of libgcc required for symbol v3.4.21 is v5.1.0

GCC 5.1.0: GLIBCXX_3.4.21, CXXABI_1.3.9

This small problem can be solved easily. I'll highlight two simple ways:

Package plan for installation in environment /conda/envs/test:

The following packages will be UPDATED:

libgcc: 4.8.5-2 --> 5.2.0-0

Proceed ([y]/n)?

(test) root@75eb43ff7f79:~$ conda list gcc

packages in environment at /conda/envs/test:

# libgcc 5.2.0 0 (test) root@75eb43ff7f79:~$ readelf --version-info /conda/envs/test/lib/libstdc++.so | grep -Po '(?<=GLIBCXX_)([\d.]*)' | sort -Vr | head -1 3.4.21 (test) root@75eb43ff7f79:~$ python test.py Reading data from table Microsoft SQL Server vNext (CTP2.0) - 14.0.500.272 (X64) Apr 13 2017 11:44:40 Copyright (C) 2017 Microsoft Corporation. All rights reserved. Developer Edition (64-bit) on Linux (Ubuntu 16.04.2 LTS)

nehaljwani commented 7 years ago

Maybe if msodbcsql was provided as a conda package, this issue wouldn't have arose, as it would have most likely taken care of the libgcc dependency. :thinking:

ijstokes commented 7 years ago

@nehaljwani amazing sleuthing work!

If we're looking for a solution here that does not involve creating an msodbcsql conda package what other options do we have available? It seems to me that documentation is the easiest, but also easiest for an end-user to miss. Another option is to provide a libgcc dependency on pyodbc that will pick up a minimum version that will work with msodbc but I don't like the sound of that since it will be imposed on everyone even if they are using an old msodbcsql or not using SQL Server at all. Are there other options you can think of? Or does it really come back around to "create an msodbcsql conda package that can specify that libgcc dependency in the right way"?

cc @ltransom

nehaljwani commented 7 years ago

The only thing I don't like about this scenario is the error handling of unixODBC. It is surprisingly, very misleading. It should have reported the correct error or at least given a better hint.

I am also against adding a higher version of libgcc as a minimum dependency for pyodbc.

xusun commented 7 years ago

Thanks, @nehaljwani . I updated libgcc, and that worked like charm! Thanks a lot for the debugging!

MaximoTrinidad commented 7 years ago

Any want to share how to fix this?? I would love to be able to try the workaround.

Thanks, Max

nehaljwani commented 7 years ago

Just wanted to update here that if python3 is being used, then 3.6.0=0 throws the error: 

(py36) root@75eb43ff7f79:~$ python test.py
Traceback (most recent call last):
  File "test.py", line 1, in <module>
    import pyodbc
ImportError: /conda/envs/py36/lib/python3.6/site-packages/pyodbc.cpython-36m-x86_64-linux-gnu.so: undefined symbol: PySlice_Unpack

This goes away when python3 is updated to something > 3.6.0

MaximoTrinidad commented 7 years ago

Hum! That's explains it.

Ubuntu 16.04.2 (I installed in Windows) has Python 3.5 before the announcement on SQL Server and Anacoda I have PYODBC already working.

And I can just change my environment from Anaconda to 3.5 and that's fine. But, I do get the error with PYODBC.

sanhitamj commented 7 years ago

I was stuck with the same problem. Thanks nehaljwani for the solution and kalefranz for asking the question (just days before me fumbling with it).

I am running Kubuntu 16.04, python 2.7.13 with conda 4.3.21.

nehaljwani commented 7 years ago

@ijstokes Do you think more discussion is required on this or can this issue be closed?

huileiw commented 7 years ago

I'm facing the same issue except I'm working on a Mac machine. There is no libgcc v5.2 package for Mac available in conda. https://anaconda.org/anaconda/libgcc/files

nehaljwani commented 7 years ago

@huileiw Could you please share the exact error message you see try to use pyodbc?

huileiw commented 7 years ago

@nehaljwani here you go. Now I realize it's not the same error. The same code works outside conda environment.

Error Traceback (most recent call last)

in () 1 import pandas as pd 2 import pyodbc ----> 3 conn = pyodbc.connect('DSN=DSN;UID=UID;PWD=PWD') 4 df = pd.read_sql("SELECT top 5 * from XXX.dbo.xxxxx", conn) Error: ('IM002', '[IM002] [unixODBC][Driver Manager]Data source name not found, and no default driver specified (0) (SQLDriverConnect)')
MaximoTrinidad commented 7 years ago

@huileiw,

Hum! It seems it might be either the driver is missing some ODBC component, or it's something in the string. I don't use DSN.

So, for testing the connection string, give the following code a try!!

If there's no errors after doing 'import pyodbc', the I use the following string format:

import pyodbc
cnxn = pyodbc.connect('DRIVER={ODBC Driver 13 for SQL Server};SERVER=serverIpAddress,portnum;DATABASE=master;UID=YourSqlAuthenUser;PWD=YourSQLAuthPasswd!')
cursor = cnxn.cursor()

#Execute T-SQL Query:
trecord = []
tsql = 'SELECT Field1, Field2, Field3, Field4 FROM database1.dbo.Table1;'
if cursor.execute(tsql):
    row = cursor.fetchone()
    while row:
        datarow = [str(row[0]),str(row[1]),str(row[2]),str(row[3])]
        trecord.append(datarow)
        row = cursor.fetchone()

## - list to screen list of data:
for i, r in enumerate(trecord):
    print(i,r);

I have use similar code to connect to both Windows and Ubuntu Linux SQL Server.

I just hope this might help in some way! :)

huileiw commented 7 years ago

thanks @MaximoTrinidad. My DSN was set up in this way (https://github.com/lionheart/django-pyodbc/wiki/Mac-setup-to-connect-to-a-MS-SQL-Server). The code works fine outside a conda environment. So I think it might be a similar problem as posted in the original thread here.

MaximoTrinidad commented 7 years ago

@huileiw,

Ah! I don't know if have done this already, few other things to verify:

  1. Make sure Anaconda is upgrade to Python 3.6.1: conda upgrade python
  2. Also, to clear my issues with pyodbc, I had to do: conda upgrade pyodbc.
  3. Add the Anaconda path in ~/.bashrc: export PATH="/home/maxt/anaconda3/bin:$PATH"

Also, in some occasions I rely in using 'sudo update-alternatives ...' to move between my different python environments.

:)

MaximoTrinidad commented 7 years ago

@huileiw,

Thanks for the link! Now I wish I had a Mac.

I think it won't hurt to also post this incident in 'Django/pyodbc' and see what they have to say. I know some Mac users are using Containers in order to work with SQL Server and Python.

So many directions you can go with it.

:)

huileiw commented 7 years ago

@MaximoTrinidad Thank you for the steps. I am actually using python 2.7 and my pyodbc is in version 4.0.16. However my .bashrc was empty. So I'll try that.

fabiopicchi commented 7 years ago

@nehaljwani Upvote for the creation of an msodbcsql package for conda.

I am trying to configure a project with anaconda-project and it is really annoying to have this external dependency of a driver. It would be very useful to add it as a dependency!

MaximoTrinidad commented 7 years ago

@nehaljwani,

That's Awesome!! It's really need to be included in conda. :) 👍

MaximoTrinidad commented 7 years ago

Hi Everyone,

Just an FYI

I went thru the process of rebuilding my Windows Bash (RTM) just now in two Windows 10 Insider Build 16257.rs3 with PowerShell Core Beta 5 and Anaconda 4.4.7 (in Bash and Windows) which includes pyodbc.

But, after the initial installation with current version of Anaconda, pyodbc was still failing in Linux: Anaconda 4.4.7 -> Installed Python 3.6.1 and pyodbc 4.0.16-py36_0 ODBC failed connection message: Can't open lib '/opt/microsoft/msodbcsql/lib64/libmsodbcsql-13.1.so.9.0'

Basically, the Windows Anaconda is very straight forward and everything seems to work. But, for Bash in Windows 10, I needed the following to clear the error: (Bash clean installation after Ubuntu Desktop)

## - Microsoft packages:
sudo apt install msodbcsql
sudo apt install mssql-tools

## - ODBC Dependencies needed:
sudo apt install unixodbc-dev
sudo apt install unixodbc-bin

## - Always good to execute:
sudo apt update
sudo apt upgrade

## - Add line and create ~/.bash_profile:
echo 'export PATH="$PATH:/opt/mssql-tools/bin"' >> ~/.bash_profile

## - Anaconda packages upgrade:
conda upgrade python
conda upgrade pyodbc
conda update libgcc

python test pyodbc connectivity to SQL Server - OK

import pyodbc

cnxn = pyodbc.connect('DRIVER={ODBC Driver 13 for SQL Server};SERVER=MyLaptop1,1338;DATABASE=master;UID=sa;PWD=@mysuppwd!')
cursor = cnxn.cursor()

Test Passed

I'm hoping this is helpful. I will be blogging on this soon. :)

consoleresults_2017-08-10_20-23-59

ijstokes commented 7 years ago

@MaximoTrinidad thanks for those details and @nehaljwani I'm considering this closed. I think we learned a lot from this.

deepix commented 7 years ago

An additional gotcha for those of you with conda environments, trying to upgrade libgcc:

Be sure to use the -n <env_name> argument! Just doing conda update libgcc==5.2.0 will not work. You should do conda update -n py35 libgcc==5.2.0, if your environment is py35.

AnkitaSambhare commented 6 years ago

Hello I am facing a similar issue... I an trying to update the location where the msodbcsql driver resides as well as where odbcinst.ini and odbc.ini files reside... odbcinst -j and odbcinst -q -d -n "ODBC Driver 11 for SQL Server" give me the right path... but the error persists:

  1. With driver = ODBC Driver 11 for SQL Server Error: pyodbc.Error: ('01000', "[01000] [unixODBC][Driver Manager]Can't open lib '/opt/microsoft/msodbcsql/lib64/libmsodbcsql-11.0.so.2270.0' : file not found (0) (SQLDriverConnect)")

  2. with driver = Error: pyodbc.Error: ('IM002', '[IM002] [unixODBC][Driver Manager]Data source name not found, and no default driver specified (0) (SQLDriverConnect)')

Can someone help?

pujy commented 5 years ago

Hello, I am also facing the similar issue. When we try to access netezza database in terminal, we were able to connect but when we try to connect in notebook, getting the below error In Notebook: Error: ('01000', "[01000] [unixODBC][Driver Manager]Can't open lib '/opt/app/nz/lib/libnzsqlodbc3.so' : file not found (0) (SQLDriverConnect)") In terminal: [pujy@server]$ source /opt/app/anaconda3/bin/activate python36 (python36) [pujy@server]$ python Python 3.6.4 |Anaconda, Inc.| (default, Mar 13 2018, 01:15:57) [GCC 7.2.0] on linux Type "help", "copyright", "credits" or "license" for more information.

import pyodbc conn = pyodbc.connect("DSN=DPR;UID=;PWD=;");

netezza version: nz-linuxclient-v7.2.0.6

Operating System: Red Hat Enterprise Linux Server release 7.3 (Maipo)

import sys print(sys.version) from notebook 3.6.4 |Anaconda, Inc.| (default, Jan 16 2018, 18:10:19) [GCC 7.2.0]

Could you please help me to debug and fix the issue?

moraisaugusto commented 5 years ago

I figured out the problem. The /opt/microsoft/msodbcsql/lib64/libmsodbcsql.x tries to load shared dependencies that aren't linked. so, try it:

$ ldd /opt/microsoft/msodbcsql/lib64/libmsodbcsql-13.1.so.3.0

response: /opt/microsoft/msodbcsql/lib64/libmsodbcsql-13.1.so.3.0: /usr/lib/libcurl.so.4: version `CURL_OPENSSL_3' not found (required by /opt/microsoft/msodbcsql/lib64/libmsodbcsql-13.1.so.3.0)

In this case you need to link the /usr/lib/libcurl.so.3 (yes, 3) because we have a CURL_OPENSSL_3 not found error.

simple workaround: export LD_PRELOAD=/usr/lib/libcurl.so.3 && python test-msodbc-connection.py