search

oracle / python-cx_Oracle

Python interface to Oracle Database now superseded by python-oracledb
https://oracle.github.io/python-cx_Oracle
Other
890 stars 361 forks source link

DPI-1072 unsupported client library when using cx_Oracle stable (8.2.1) + instant client basic 19.11.0 (Win 64 bit) #576

Closed nicolasap-dm closed 3 years ago

nicolasap-dm commented 3 years ago

  1. What versions are you using?

    platform.platform: Windows-10-10.0.18362-SP0
sys.maxsize > 2**32: True
platform.python_version: 3.6.13

cx_Oracle.version: 8.2.1

  2. Is it an error or a hang or a crash?

    Error

  3. What error(s) or behavior you are seeing?

    I can't initialize the client. I get "DPI-1072: the Oracle Client library version is unsupported".

    >>> cx_Oracle.init_oracle_client(lib_dir=r"C:\oracle\instantclient_19_11")
ODPI [08484] 2021-07-07 10:50:25.266: Context Parameters:
ODPI [08484] 2021-07-07 10:50:25.267:     Oracle Client Lib Dir: C:\oracle\instantclient_19_11
ODPI [08484] 2021-07-07 10:50:25.270: Environment Variables:
ODPI [08484] 2021-07-07 10:50:25.271:     PATH => "C:\Users\nicola\.edm\envs\mwe;C:\Users\nicola\.edm\envs\mwe\Scripts;C:\Windows\system32;C:\Windows;C:\Windows\System32\Wbem;C:\Windows\System32\WindowsPowerShell\v1.0\;C:\Windows\System32\OpenSSH\;C:\Program Files\Git\cmd;C:\Users\nicola\AppData\Local\Programs\Enthought\edm\;C:\Users\nicola\AppData\Local\Microsoft\WindowsApps;"
ODPI [08484] 2021-07-07 10:50:25.285: load in parameter directory
ODPI [08484] 2021-07-07 10:50:25.287: load in dir C:\oracle\instantclient_19_11 
ODPI [08484] 2021-07-07 10:50:25.289: load with name C:\oracle\instantclient_19_11/oci.dll
ODPI [08484] 2021-07-07 10:50:25.292: load by OS successful
ODPI [08484] 2021-07-07 10:50:25.297: validating loaded library
Traceback (most recent call last):
 File "<stdin>", line 1, in <module>
cx_Oracle.DatabaseError: DPI-1072: the Oracle Client library version is unsupported

    I'm using the oracle client version that is recommended in the documentation here (e.g. the archive file is called the same), and I have the latest VS C++ redistributable installed.

    >>> # sanity check
... import os; print(os.path.isdir(r"C:\oracle\instantclient_19_11"))
True
>>> # sanity check
... with open(r"C:\oracle\instantclient_19_11\BASIC_README", "r") as fp:
...     print([next(fp) for _ in range(5)])
...
['Basic Package Information \n', '========================= \n', 'Sat Apr 17 18:29:12 MDT 2021\n', 'Client Shared Library 64-bit - 19.11.0.0.0\n', '\n']

  4. Include a runnable Python script that shows the problem.

    import cx_Oracle
cx_Oracle.init_oracle_client(lib_dir=r"C:\oracle\instantclient_19_11")
cjbj commented 3 years ago

It just sounds like a clash somewhere - or the VS redistributable isn't correct.

Do you have other oci.dll's on the system? What other Oracle software is installed?

nicolasap-dm commented 3 years ago

No other Oracle software as far as I'm aware: this is a rather fresh VM. I also cannot find any oci.dll (only some mtxoci.dll's in system folders)

The redistributable is the one found at this link: https://support.microsoft.com/en-us/help/2977003/the-latest-supported-visual-c-downloads that corresponds to my bitness (64)

nicolasap-dm commented 3 years ago

Note: I just found that it works correctly if the library is provided via the PATH:

In [1]: import os, cx_Oracle

In [2]: cx_Oracle.init_oracle_client()
---------------------------------------------------------------------------
DatabaseError                             Traceback (most recent call last)
<ipython-input-2-8ebdf9c3f24d> in <module>
----> 1 cx_Oracle.init_oracle_client()

DatabaseError: DPI-1047: Cannot locate a 64-bit Oracle Client library: "The specified module could not be found".
See https://cx-oracle.readthedocs.io/en/latest/user_guide/installation.html for help

(initially the client is not in the path. That's expected)

In [3]: cx_Oracle.init_oracle_client(r"C:\oracle\instantclient_19_11")
---------------------------------------------------------------------------
DatabaseError                             Traceback (most recent call last)
<ipython-input-3-0de1d59abba9> in <module>
----> 1 cx_Oracle.init_oracle_client(r"C:\oracle\instantclient_19_11")

DatabaseError: DPI-1072: the Oracle Client library version is unsupported

↑ this is the (in my opinion) possible bug. I would expect this to work.

In [4]: os.environ["PATH"] = r"C:\oracle\instantclient_19_11;" + os.environ["PATH"]

In [5]: cx_Oracle.init_oracle_client()

In [6]: cx_Oracle.clientversion()
Out[6]: (19, 11, 0, 0, 0)

↑ at least, the alternative route (adding via PATH) works. Of course I would normally do it from the OS/shell, not from Python, but this is for demonstration only.

This solves my use case as a workaround, but should't cx_Oracle.init_oracle_client(path) work too?

anthony-tuininga commented 3 years ago

That suggests rather strongly that there is something in PATH that is interfering. When you use cx_Oracle.init_oracle_client() the driver simply attempts to load the library at the exact location specified -- but it seems like something in PATH is preventing a successful load. Set the environment variable DPI_DEBUG_LEVEL to the value 72 so we can see a few more details about the error that is being raised. That might shed some light on what the problem is.

Currently ODPI-C is simply using LoadLibrary() on Windows. It might be necessary to use LoadLibraryEx() with some flags. See here for details. If you are able to build yourself and adjust the call in odpi/src/dpiOci.c from LoadLibrary() to LoadLibraryEx() with LOAD_LIBRARY_SEARCH_DLL_LOAD_DIR and see if that makes a difference for you? I am not overly familiar with Windows so hopefully you are knowledgeable or someone who is chimes in here! You're the first person to note this issue so it would be good to know the reason for the problem so we can update our documentation at least.

nicolasap-dm commented 3 years ago

Thanks for looking into this! I'm surprised that something may be conflicting, since I'm using a virtual machine with almost vanilla Windows 10, with the addition of a git client, a Python environment manager, and almost nothing else.

Everything else in my path:

 'C:\\Users\\nicola\\.edm\\envs\\TRIAL_ENV\\lib\\site-packages\\pywin32_system32;
C:\\Users\\nicola\\.edm\\envs\\TRIAL_ENV;
C:\\Users\\nicola\\.edm\\envs\\TRIAL_ENV\\Scripts;
C:\\WINDOWS\\system32;
C:\\WINDOWS;
C:\\WINDOWS\\System32\\Wbem;
C:\\WINDOWS\\System32\\WindowsPowerShell\\v1.0\\;
C:\\WINDOWS\\System32\\OpenSSH\\;
C:\\Program Files\\Git\\cmd;
C:\\Users\\nicola\\AppData\\Local\\Programs\\Enthought\\edm\\;
C:\\Users\\nicola\\AppData\\Local\\Microsoft\\WindowsApps;'

With debug codes

Failing init from explicit location:

>>> cx_Oracle.init_oracle_client(r"C:\oracle\instantclient_19_11")

ODPI [06780] 2021-07-15 09:00:44.926: Context Parameters:
ODPI [06780] 2021-07-15 09:00:44.926:     Oracle Client Lib Dir: C:\oracle\instantclient_19_11
ODPI [06780] 2021-07-15 09:00:44.926: Environment Variables:
ODPI [06780] 2021-07-15 09:00:44.926:     PATH => "C:\Users\nicola\.edm\envs\TRIAL_ENV\lib\site-packages\pywin32_system32;C:\Users\nicola\.edm\envs\TRIAL_ENV;C:\Users\nicola\.edm\envs\TRIAL_ENV\Scripts;C:\WINDOWS\system32;C:\WINDOWS;C:\WINDOWS\System32\Wbem;C:\WINDOWS\System32\WindowsPowerShell\v1.0\;C:\WINDOWS\System32\OpenSSH\;C:\Program Files\Git\cmd;C:\Users\nicola\AppData\Local\Programs\Enthought\edm\;C:\Users\nicola\AppData\Local\Microsoft\WindowsApps;"
ODPI [06780] 2021-07-15 09:00:44.926: load in parameter directory
ODPI [06780] 2021-07-15 09:00:44.926: load in dir C:\oracle\instantclient_19_11
ODPI [06780] 2021-07-15 09:00:44.926: load with name C:\oracle\instantclient_19_11/oci.dll
ODPI [06780] 2021-07-15 09:00:44.926: load by OS successful
ODPI [06780] 2021-07-15 09:00:44.926: validating loaded library
ODPI [06780] 2021-07-15 09:00:44.942: internal error DPI-1072: the Oracle Client library version is unsupported (dpiContext_createWithParams / get OCI client version)
---------------------------------------------------------------------------
DatabaseError                             Traceback (most recent call last)
<ipython-input-4-0de1d59abba9> in <module>
----> 1 cx_Oracle.init_oracle_client(r"C:\oracle\instantclient_19_11")

DatabaseError: DPI-1072: the Oracle Client library version is unsupported

Succesful init from the heuristics (after having added the client to the path):

>>> cx_Oracle.init_oracle_client()

ODPI [07096] 2021-07-15 09:03:37.762: Context Parameters:
ODPI [07096] 2021-07-15 09:03:37.762: Environment Variables:
ODPI [07096] 2021-07-15 09:03:37.762:     PATH => "C:\Users\nicola\.edm\envs\TRIAL_ENV\lib\site-packages\pywin32_system32;C:\Users\nicola\.edm\envs\TRIAL_ENV;C:\Users\nicola\.edm\envs\TRIAL_ENV\Scripts;C:\oracle\instantclient_19_11;C:\WINDOWS\system32;C:\WINDOWS;C:\WINDOWS\System32\Wbem;C:\WINDOWS\System32\WindowsPowerShell\v1.0\;C:\WINDOWS\System32\OpenSSH\;C:\Program Files\Git\cmd;C:\oracle\instantclient_19_11;C:\Users\nicola\AppData\Local\Programs\Enthought\edm\;C:\Users\nicola\AppData\Local\Microsoft\WindowsApps;"
ODPI [07096] 2021-07-15 09:03:37.762: check module directory
ODPI [07096] 2021-07-15 09:03:37.762: module name is C:\Users\nicola\.edm\envs\TRIAL_ENV\lib\site-packages\cx_Oracle.cp36-win_amd64.pyd
ODPI [07096] 2021-07-15 09:03:37.783: load in dir C:\Users\nicola\.edm\envs\TRIAL_ENV\lib\site-packages
ODPI [07096] 2021-07-15 09:03:37.783: load with name C:\Users\nicola\.edm\envs\TRIAL_ENV\lib\site-packages/oci.dll
ODPI [07096] 2021-07-15 09:03:37.783: load by OS failure: The specified module could not be found
ODPI [07096] 2021-07-15 09:03:37.783: load with OS search heuristics
ODPI [07096] 2021-07-15 09:03:37.783: load with name oci.dll
ODPI [07096] 2021-07-15 09:03:37.783: load by OS successful
ODPI [07096] 2021-07-15 09:03:37.783: validating loaded library

Unfortunately I'm not able to build cx_Oracle at present, sorry (and I also only use Windows occasionally for testing :slightly_smiling_face:)

anthony-tuininga commented 3 years ago

One other thing to attempt: set PATH to just the minimum necessary (but without the instant client directory) and then call init_oracle_client(instant_client_install_dir) and see if the problem goes away. If it does, you can then find out which element of the PATH is causing the conflict by adding them one at a time until the problem recurs. This may be helpful in resolving the issue permanently. I'll ask internally if there is another way to diagnose the problem. Thanks!

pvenkatraman commented 3 years ago

Looks like some dependent DLL is not in the PATH and while loading OCI.DLL, it fails as the dependent DLL can not be loaded. The best way to find out which DLL is to use ntsd.exe or windbg (Windows debugging tools 64 bit) and run your python script, and when it fails, you can see the error message about "unable to load xxx.DLL " error message.

nicolasap-dm commented 3 years ago

Thanks both.

set PATH to just the minimum necessary

The issue persists even with basically nothing in the path

use ntsd.exe or windbg

Trying that soon

anthony-tuininga commented 3 years ago

Another suggestion: check the Windows event log. There might be something useful there as an event is logged when something like this happens. If you find something there, please let us know! Thanks.

anthony-tuininga commented 3 years ago

And try with aboslutely nothing in PATH as well.

nicolasap-dm commented 3 years ago

Tried with nothing in PATH: same behaviour

In [2]: os.environ["PATH"] = ""

In [3]: cx_Oracle.init_oracle_client(r"C:\oracle\instantclient_19_11")
ODPI [06796] 2021-07-19 08:26:09.936: Context Parameters:
ODPI [06796] 2021-07-19 08:26:09.985:     Oracle Client Lib Dir: C:\oracle\instantclient_19_11
ODPI [06796] 2021-07-19 08:26:10.020: Environment Variables:
ODPI [06796] 2021-07-19 08:26:10.062: load in parameter directory
ODPI [06796] 2021-07-19 08:26:10.078: load in dir C:\oracle\instantclient_19_11
ODPI [06796] 2021-07-19 08:26:10.108: load with name C:\oracle\instantclient_19_11/oci.dll
ODPI [06796] 2021-07-19 08:26:10.218: load by OS successful
ODPI [06796] 2021-07-19 08:26:10.265: validating loaded library
ODPI [06796] 2021-07-19 08:26:10.328: internal error DPI-1072: the Oracle Client library version is unsupported (dpiContext_createWithParams / get OCI client version)
---------------------------------------------------------------------------
DatabaseError                             Traceback (most recent call last)
<ipython-input-3-0de1d59abba9> in <module>
----> 1 cx_Oracle.init_oracle_client(r"C:\oracle\instantclient_19_11")

DatabaseError: DPI-1072: the Oracle Client library version is unsupported

I'm not generally a Windows user so I'm definitely not too familiar with its debugging tools. I don't seem to see any event in the Event Viewer (under either System, Application, Setup, ...) with a matching timestamp to my cx_Oracle attempts. As for using the Windows Debugger or Ntsd, I'd need more specific guidance.

nicolasap-dm commented 3 years ago

For the sake of reproducibility, I just run with the standard CPython distribution (both Python 3.6 and 3.9): the issue doesn't appear. It is therefore currently only observed with Enthought EDM's Python. I'll check internally what issue/cross-wiring Enthought Python may be introducing.

In the meanwhile, I'm fine if this issue gets closed.

anthony-tuininga commented 3 years ago

Good to know! I'll go ahead and close this, then. Please re-open or create a new issue if you discover the source of the issue!

alvinwanda commented 2 years ago

Note: I just found that it works correctly if the library is provided via the PATH:

In [1]: import os, cx_Oracle

In [2]: cx_Oracle.init_oracle_client()
---------------------------------------------------------------------------
DatabaseError                             Traceback (most recent call last)
<ipython-input-2-8ebdf9c3f24d> in <module>
----> 1 cx_Oracle.init_oracle_client()

DatabaseError: DPI-1047: Cannot locate a 64-bit Oracle Client library: "The specified module could not be found".
See https://cx-oracle.readthedocs.io/en/latest/user_guide/installation.html for help

(initially the client is not in the path. That's expected)

In [3]: cx_Oracle.init_oracle_client(r"C:\oracle\instantclient_19_11")
---------------------------------------------------------------------------
DatabaseError                             Traceback (most recent call last)
<ipython-input-3-0de1d59abba9> in <module>
----> 1 cx_Oracle.init_oracle_client(r"C:\oracle\instantclient_19_11")

DatabaseError: DPI-1072: the Oracle Client library version is unsupported

↑ this is the (in my opinion) possible bug. I would expect this to work.

In [4]: os.environ["PATH"] = r"C:\oracle\instantclient_19_11;" + os.environ["PATH"]

In [5]: cx_Oracle.init_oracle_client()

In [6]: cx_Oracle.clientversion()
Out[6]: (19, 11, 0, 0, 0)

↑ at least, the alternative route (adding via PATH) works. Of course I would normally do it from the OS/shell, not from Python, but this is for demonstration only.

This solves my use case as a workaround, but should't cx_Oracle.init_oracle_client(path) work too?

For anyone referencing this in the near or distant future, the last snippet works because it appends the instant_client at the beginning of PATH, whereas more popular techniques (like using set PATH from OS/shell) append instant_client at the end of PATH. remember to add ";" at the end of the string.