search

lurcher / unixODBC

The unixODBC Project goals are to develop and promote unixODBC to be the definitive standard for ODBC on non MS Windows platforms.
GNU Lesser General Public License v2.1
103 stars 52 forks source link

I'm having weird issues with pyodbc and I *think* it's because of odbc_config #164

Open guymatz opened 8 months ago

guymatz commented 8 months ago

pyodbc is complaining that it can't find my libs, but strace tells me that it can, but it can't find libodbcinst . . .

odbc_config --libs returns -L/path/to/libs -lodbc . . . should it return -L/path/to/libs -lodbc -lodbcinst?

lurcher commented 8 months ago

On 17/03/2024 00:37, Guy Matz wrote:

pyodbc is complaining that it can't find my libs, but strace tells me that it can, but it can't find libodbcinst . . .

|odbc_config --libs| returns `-L/path/to/libs -lodbc|. . . should it return|-L/path/to/libs -lodbc -lodbcinst?

— Reply to this email directly, view it on GitHub https://github.com/lurcher/unixODBC/issues/164, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABYK62I3TH7HJPTTWKMGBSTYYTQVNAVCNFSM6AAAAABEZXQ5OOVHI2DSMVQWIX3LMV43ASLTON2WKOZSGE4TANBQGI2DKOI. You are receiving this because you are subscribed to this thread.Message ID: @.***>

The distribution builds a so.2 lib, make sure with ldd that pyodbc is not looking for a so.1 If so a rename or link should do the job.

guymatz commented 8 months ago

Thanks so much for the reply!!

ldd pyodbc.cpython-37m-x86_64-linux-gnu.so  | grep odbc
        libodbc.so.2 => /opt/sw/lib64/libodbc.so.2 (0x00007f960f301000)

pyodbc works when I have LD_LIBRARY_PATH set to the dir of libodbc & friends . . . . it seems that the only difference between when I have LD_LIBRARY_PATH set and when I don't - I.e., when pyodbc works and doesn't - is that libodbcinst is found when LD_LIBRARY_PATH is set: open("/opt/sw/lib64/libodbcinst.so.2", O_RDONLY|O_CLOEXEC) = 3

When LD_LIBRARY_PATH is not set, pyodbc fails with pyodbc.Error: ('01000', "[01000] [unixODBC][Driver Manager]Can't open lib ' /opt/sw/microsoft/msodbcsql18/lib64/libmsodbcsql-18.2.so.1.1' : file not found (0) (SQLDriverConnect)")

However strace is telling me that libmsodbcsql is available (not-works.out & works.out are strace output files): $ grep libmsodbcsql-18 not-works.out works.out
not-works.out:write(1, "conn str: DRIVER={/opt/bb/microsoft/msodbcsql18/lib64/libmsodbcsql-18.2.so.1.1};SERVER=XXX,XXX;DATABASE=XXX;UID=XXX;PWD=XXX;Encrypt=yes;TrustServerCertificate=yes\n", 186) = 186
not-works.out:open("/opt/bb/microsoft/msodbcsql18/lib64/libmsodbcsql-18.2.so.1.1", O_RDONLY|O_CLOEXEC) = 3 not-works.out:open("/opt/bb/microsoft/msodbcsql18/lib64/libmsodbcsql-18.2.so.1.1", O_RDONLY|O_CLOEXEC) = 3 not-works.out:write(2, "pyodbc.Error: ('01000', \"[01000] [unixODBC][Driver Manager]Can't open lib '/opt/sw/microsoft/msodbcsql18/lib64/libmsodbcsql-18.2.so.1.1' : file not found (0) (SQLDriverConnect)\")\n", 179) = 179

works.out:write(1, "conn str: DRIVER={/opt/sw/microsoft/msodbcsql18/lib64/libmsodbcsql-18.2.so.1.1};SERVER=XXX,XXX;DATABASE=XXX;UID=XXX;PWD=XXX;Encrypt=yes;TrustServerCertificate=yes\n", 186) = 186 works.out:open("/opt/sw/microsoft/msodbcsql18/lib64/libmsodbcsql-18.2.so.1.1", O_RDONLY|O_CLOEXEC) = 3

And I see this sort of thing in strace: open("/lib64/tls/x86_64/libodbcinst.so.2", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory) open("/lib64/tls/libodbcinst.so.2", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory) open("/lib64/x86_64/libodbcinst.so.2", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory) open("/lib64/libodbcinst.so.2", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
open("/usr/lib64/tls/x86_64/libodbcinst.so.2", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory) open("/usr/lib64/tls/libodbcinst.so.2", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)

Am I barking up the wrong tree here, thinking the problem is with libodbcinst? I feel like line 1186 of DriverManager/SQLConnect.c is lying to me - that the underlying problem is with finding libodbcinst - but my C is not good enough to figure out what's going on there.

Thank you soooooo much!

lurcher commented 8 months ago

On 17/03/2024 16:36, Guy Matz wrote:

Thanks so much for the reply!!

|ldd pyodbc.cpython-37m-x86_64-linux-gnu.so | grep odbc libodbc.so.2 => /opt/sw/lib64/libodbc.so.2 (0x00007f960f301000) |

pyodbc works when I have LD_LIBRARY_PATH set to the dir of libodbc & friends . . . . it /seems/ that the only difference between when I have LD_LIBRARY_PATH set and when I don't - I.e., when pyodbc works and doesn't - is that libodbcinst is found when LD_LIBRARY_PATH is set: open("/opt/sw/lib64/libodbcinst.so.2", O_RDONLY|O_CLOEXEC) = 3

When LD_LIBRARY_PATH is not set, pyodbc fails with pyodbc.Error: ('01000', "[01000] [unixODBC][Driver Manager]Can't open lib ' /opt/sw/microsoft/msodbcsql18/lib64/libmsodbcsql-18.2.so.1.1' : file not found (0) (SQLDriverConnect)")

However strace is telling me that libmsodbcsql is available (not-works.out & works.out are strace output files): $ grep libmsodbcsql-18 not-works.out works.out not-works.out:write(1, "conn str: DRIVER={/opt/bb/microsoft/msodbcsql18/lib64/libmsodbcsql-18.2.so.1.1};SERVER=XXX,XXX;DATABASE=XXX;UID=XXX;PWD=XXX;Encrypt=yes;TrustServerCertificate=yes\n", 186) = 186 not-works.out:open("/opt/bb/microsoft/msodbcsql18/lib64/libmsodbcsql-18.2.so.1.1", O_RDONLY|O_CLOEXEC) = 3 not-works.out:open("/opt/bb/microsoft/msodbcsql18/lib64/libmsodbcsql-18.2.so.1.1", O_RDONLY|O_CLOEXEC) = 3 not-works.out:write(2, "pyodbc.Error: ('01000', "[01000] [unixODBC][Driver Manager]Can't open lib '/opt/sw/microsoft/msodbcsql18/lib64/libmsodbcsql-18.2.so.1.1' : file not found (0) (SQLDriverConnect)")\n", 179) = 179

works.out:write(1, "conn str: DRIVER={/opt/sw/microsoft/msodbcsql18/lib64/libmsodbcsql-18.2.so.1.1};SERVER=XXX,XXX;DATABASE=XXX;UID=XXX;PWD=XXX;Encrypt=yes;TrustServerCertificate=yes\n", 186) = 186 works.out:open("/opt/sw/microsoft/msodbcsql18/lib64/libmsodbcsql-18.2.so.1.1", O_RDONLY|O_CLOEXEC) = 3

And I see this sort of thing in strace: open("/lib64/tls/x86_64/libodbcinst.so.2", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory) open("/lib64/tls/libodbcinst.so.2", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory) open("/lib64/x86_64/libodbcinst.so.2", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory) open("/lib64/libodbcinst.so.2", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory) open("/usr/lib64/tls/x86_64/libodbcinst.so.2", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory) open("/usr/lib64/tls/libodbcinst.so.2", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)

Am I barking up the wrong tree here, thinking the problem is with libodbcinst? I /feel/ like line 1186 of DriverManager/SQLConnect.c is lying to me - that the underlying problem is with finding libodbcinst

  • but my C is not good enough to figure out what's going on there.

Thank you soooooo much!

Try

ldd /opt/bb/microsoft/msodbcsql18/lib64/libmsodbcsql-18.2.so.1.1

To see what the driver is linked against. I suspect you will find it cant find the lib unless LD_LIBRARY_PATH is set. Which if its somewhere not in the standard list of places like /opt/sw/lib64 I would expect.

guymatz commented 8 months ago

Ahhh!!!! /opt/bb/microsoft/msodbcsql18/lib64/libmsodbcsql-18.2.so.1.1
linux-vdso.so.1 => (0x00007ffd1cd91000) libdl.so.2 => /lib64/libdl.so.2 (0x00007f33b3987000) librt.so.1 => /lib64/librt.so.1 (0x00007f33b377f000) libodbcinst.so.2 => not found
libkrb5.so.3 => /lib64/libkrb5.so.3 (0x00007f33b3496000)
libgssapi_krb5.so.2 => /lib64/libgssapi_krb5.so.2 (0x00007f33b3249000) libstdc++.so.6 => /lib64/libstdc++.so.6 (0x00007f33b2f41000)
libm.so.6 => /lib64/libm.so.6 (0x00007f33b2c3f000) libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x00007f33b2a29000) libpthread.so.0 => /lib64/libpthread.so.0 (0x00007f33b280d000) libc.so.6 => /lib64/libc.so.6 (0x00007f33b243f000) /lib64/ld-linux-x86-64.so.2 (0x00007f33b3f9b000) libk5crypto.so.3 => /lib64/libk5crypto.so.3 (0x00007f33b220c000) libcom_err.so.2 => /lib64/libcom_err.so.2 (0x00007f33b2008000) libkrb5support.so.0 => /lib64/libkrb5support.so.0 (0x00007f33b1df8000) libkeyutils.so.1 => /lib64/libkeyutils.so.1 (0x00007f33b1bf4000) libresolv.so.2 => /lib64/libresolv.so.2 (0x00007f33b19da000) libselinux.so.1 => /lib64/libselinux.so.1 (0x00007f33b17b3000) libpcre.so.1 => /lib64/libpcre.so.1 (0x00007f33b1551000)

Thank you!!!!! I guess I'll have to talk to the people who packaged msodbcsql18 . . . any further advice? Thanks again!

lurcher commented 8 months ago

On 17/03/2024 18:08, Guy Matz wrote:

Ahhh!!!! /opt/bb/microsoft/msodbcsql18/lib64/libmsodbcsql-18.2.so.1.1 linux-vdso.so.1 => (0x00007ffd1cd91000) libdl.so.2 => /lib64/libdl.so.2 (0x00007f33b3987000) librt.so.1 => /lib64/librt.so.1 (0x00007f33b377f000) libodbcinst.so.2 => not found libkrb5.so.3 => /lib64/libkrb5.so.3 (0x00007f33b3496000) libgssapi_krb5.so.2 => /lib64/libgssapi_krb5.so.2 (0x00007f33b3249000) libstdc++.so.6 => /lib64/libstdc++.so.6 (0x00007f33b2f41000) libm.so.6 => /lib64/libm.so.6 (0x00007f33b2c3f000) libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x00007f33b2a29000) libpthread.so.0 => /lib64/libpthread.so.0 (0x00007f33b280d000) libc.so.6 => /lib64/libc.so.6 (0x00007f33b243f000) /lib64/ld-linux-x86-64.so.2 (0x00007f33b3f9b000) libk5crypto.so.3 => /lib64/libk5crypto.so.3 (0x00007f33b220c000) libcom_err.so.2 => /lib64/libcom_err.so.2 (0x00007f33b2008000) libkrb5support.so.0 => /lib64/libkrb5support.so.0 (0x00007f33b1df8000) libkeyutils.so.1 => /lib64/libkeyutils.so.1 (0x00007f33b1bf4000) libresolv.so.2 => /lib64/libresolv.so.2 (0x00007f33b19da000) libselinux.so.1 => /lib64/libselinux.so.1 (0x00007f33b17b3000) libpcre.so.1 => /lib64/libpcre.so.1 (0x00007f33b1551000)

Thank you!!!!! I guess I'll have to talk to the people who packaged msodbcsql18 . . . any further advice? Thanks again!

Either set LD_LIBRARY_PATH, Add the path to /etc/ld.so.conf or move the libs to somewhere that is already in the search path for libs.

guymatz commented 8 months ago

Thanks again. Last question! Do you know if there is a way around requiring libodbcinst? Some env var I can set (besides LD_LIBRARY_PATH, which needs to be set before I start python, which I'm trying to avoid, because I don't want a wrapper script)? Or something I can set in my connection string? I'll do anything to avoid a wrapper script!!

P.S. - My SAs don't want to help, including either adding to ld.so.conf or mv'ing the lib :-(

v-chojas commented 8 months ago

You can set environment variables from within the python too. https://docs.python.org/3/library/os.html#os.environ

guymatz commented 8 months ago

Doesn't work for LD_LIBRARY_PATH :-( https://stackoverflow.com/questions/856116/changing-ld-library-path-at-runtime-for-ctypes