Cannot find libclntsh.so when using python 3.10 on Linux

[pgp]

1. What versions are you using?

• cx_Oracle 8.3.0
• Linux Mint 20 x64
• Python 3.10 (from miniconda)
• Instantclient version (exported in LD_LIBRARY_PATH before running python): 21.4

platform.platform: Linux-5.10.0-051000-generic-x86_64-with-glibc2.31
sys.maxsize > 2**32: True
platform.python_version: 3.10.0

---------------------------------------------------------------------------
DatabaseError                             Traceback (most recent call last)
<ipython-input-1-5bbb073c451f> in <module>
      1 import cx_Oracle
      2 print("cx_Oracle.version:", cx_Oracle.version)
----> 3 print("cx_Oracle.clientversion:", cx_Oracle.clientversion())

DatabaseError: DPI-1047: Cannot locate a 64-bit Oracle Client library: "libclntsh.so: cannot open shared object file: No such file or directory". See https://cx-oracle.readthedocs.io/en/latest/user_guide/installation.html for help

2. Is it an error or a hang or a crash? A crash

3. What error(s) or behavior you are seeing? cx_Oracle doesn't find instantclient, even if its directory path is correctly exported in LD_LIBRARY_PATH

The problem doesn't appear with previous python versions (tested both with 3.8, 3.9, in different conda environments) while keeping both the same instantclient binaries and using the same cx_Oracle version (8.3.0)

Instantclient binaries were downloaded from: https://download.oracle.com/otn_software/linux/instantclient/214000/instantclient-basic-linux.x64-21.4.0.0.0dbru.zip

[anthony-tuininga]

Set the environment variable DPI_DEBUG_LEVEL to the value 64 and post the results. This should help figure out what is going wrong with your setup.

[pgp]

ODPI [32184] 2021-11-26 15:47:47.156: debugging messages initialized at level 64

< user application proceeds >

ODPI [32184] 2021-11-26 15:48:18.131: Context Parameters:
ODPI [32184] 2021-11-26 15:48:18.131: Environment Variables:
ODPI [32184] 2021-11-26 15:48:18.131: check module directory
ODPI [32184] 2021-11-26 15:48:18.131: module name is /home/pgp/miniconda3/envs/py310env/lib/python3.10/site-packages/cx_Oracle.cpython-310-x86_64-linux-gnu.so
ODPI [32184] 2021-11-26 15:48:18.131: load in dir /home/pgp/miniconda3/envs/py310env/lib/python3.10/site-packages
ODPI [32184] 2021-11-26 15:48:18.131: load with name /home/pgp/miniconda3/envs/py310env/lib/python3.10/site-packages/libclntsh.so
ODPI [32184] 2021-11-26 15:48:18.131: load by OS failure: /home/pgp/miniconda3/envs/py310env/lib/python3.10/site-packages/libclntsh.so: cannot open shared object file: No such file or directory
ODPI [32184] 2021-11-26 15:48:18.131: load with OS search heuristics
ODPI [32184] 2021-11-26 15:48:18.131: load with name libclntsh.so
ODPI [32184] 2021-11-26 15:48:18.131: load by OS failure: libclntsh.so: cannot open shared object file: No such file or directory
ODPI [32184] 2021-11-26 15:48:18.131: load with name libclntsh.so.19.1
ODPI [32184] 2021-11-26 15:48:18.131: load by OS failure: libclntsh.so.19.1: cannot open shared object file: No such file or directory
ODPI [32184] 2021-11-26 15:48:18.131: load with name libclntsh.so.18.1
ODPI [32184] 2021-11-26 15:48:18.131: load by OS failure: libclntsh.so.18.1: cannot open shared object file: No such file or directory
ODPI [32184] 2021-11-26 15:48:18.131: load with name libclntsh.so.12.1
ODPI [32184] 2021-11-26 15:48:18.131: load by OS failure: libclntsh.so.12.1: cannot open shared object file: No such file or directory
ODPI [32184] 2021-11-26 15:48:18.131: load with name libclntsh.so.11.1
ODPI [32184] 2021-11-26 15:48:18.131: load by OS failure: libclntsh.so.11.1: cannot open shared object file: No such file or directory
ODPI [32184] 2021-11-26 15:48:18.131: load with name libclntsh.so.20.1
ODPI [32184] 2021-11-26 15:48:18.131: load by OS failure: libclntsh.so.20.1: cannot open shared object file: No such file or directory
ODPI [32184] 2021-11-26 15:48:18.131: load with name libclntsh.so.21.1
ODPI [32184] 2021-11-26 15:48:18.131: load by OS failure: libclntsh.so.21.1: cannot open shared object file: No such file or directory
ODPI [32184] 2021-11-26 15:48:18.131: check ORACLE_HOME

<user application stacktrace, crash upon  cx_Oracle.connect(user=user, password=password, dsn=dsn)>

cx_Oracle.DatabaseError: DPI-1047: Cannot locate a 64-bit Oracle Client library: "libclntsh.so: cannot open shared object file: No such file or directory". See https://cx-oracle.readthedocs.io/en/latest/user_guide/installation.html for help

[pgp]

Using a simplified script like this:

import os

print('BEFORE:\n'+ '*'*100+'\n'+os.getenv('LD_LIBRARY_PATH', 'None')+'\n'+'*'*100)

import cx_Oracle

print('AFTER:\n'+ '*'*100+'\n'+os.getenv('LD_LIBRARY_PATH', 'None')+'\n'+'*'*100)

and invoking it with:

export LD_LIBRARY_PATH=/home/pgp/opt/instantclient_21_4
python example.py

Outputs this in the python 3.8 env:

BEFORE:
****************************************************************************************************
/home/pgp/opt/instantclient_21_4
****************************************************************************************************
AFTER:
****************************************************************************************************
/home/pgp/opt/instantclient_21_4
****************************************************************************************************

And this for python 3.10:

BEFORE:
****************************************************************************************************
None
****************************************************************************************************
AFTER:
****************************************************************************************************
None
****************************************************************************************************

So, it seems the python interpreter itself in version 3.10 is resetting the LD_LIBRARY_PATH variable from the beginning...

[anthony-tuininga]

From the output of DPI_DEBUG_LEVEL=64 it is clear that the environment variable LD_LIBRARY_PATH is not actually set. One possibility is that whatever you are executing to run Python 3.10 itself clears the environment and then either re-executes itself or spawns a different executable. I don't have that happening on my machine -- but I am not using miniconda. Perhaps try without miniconda?

[pgp]

Using a Docker container (python:3.10-slim) the LD_LIBRARY_PATH is correctly recognized, and connection works... So, it definitely must be an Anaconda issue...

[anthony-tuininga]

That's good to hear. I'll close this, then, since it isn't related to cx_Oracle. If you have further questions, however, feel free to ask!

[cjbj]

Issues like this are why we generally recommend using ldconfig instead of setting LD_LIBRARY_PATH. See the Instant Client (and cx_Oracle) installation instructions.

[pgp]

btw, maybe it was due to a malformed conda environment (didn't find the exact cause though). I recreated the env with same python version and installed same requirements, and now everything works fine

[abdelouahabb]

this solution worked with me manjaro.site/how-to-install-oracle-instant-client-on-ubuntu-20-04/

[cjbj]

@abdelouahabb Thanks for the feedback. Luckily those instructions are effectively the same as our installation manual: https://cx-oracle.readthedocs.io/en/latest/user_guide/installation.html#oracle-instant-client-zip-files
(Since your link is a bit spammy, I've removed the protocol so there isn't any auto link creation)