oracle / python-oracledb

Python driver for Oracle Database conforming to the Python DB API 2.0 specification. This is the renamed, new major release of cx_Oracle
https://oracle.github.io/python-oracledb
Other
307 stars 59 forks source link

Extrange case, do not work on server (error DPY-4011) but inside docker container same server works ok #315

Closed i-m-d closed 2 months ago

i-m-d commented 3 months ago

Hello,

I have a strange case with this simple code:

import oracledb connection = oracledb.connect( user=myuser, password=mypass, dsn=mydsn)

this simple code executed in script or python3 shell do not work:

Traceback (most recent call last): File "src/oracledb/impl/thin/transport.pyx", line 312, in oracledb.thin_impl.Transport.read_packet ConnectionResetError: [Errno 104] Connection reset by peer

The above exception was the direct cause of the following exception:

Traceback (most recent call last): File "/home/rancid/pp.py", line 2, in connection = oracledb.connect( File "/home/rancid/.local/lib/python3.9/site-packages/oracledb/connection.py", line 1158, in connect return conn_class(dsn=dsn, pool=pool, params=params, **kwargs) File "/home/rancid/.local/lib/python3.9/site-packages/oracledb/connection.py", line 541, in init impl.connect(params_impl) File "src/oracledb/impl/thin/connection.pyx", line 381, in oracledb.thin_impl.ThinConnImpl.connect File "src/oracledb/impl/thin/connection.pyx", line 377, in oracledb.thin_impl.ThinConnImpl.connect File "src/oracledb/impl/thin/connection.pyx", line 337, in oracledb.thin_impl.ThinConnImpl._connect_with_params File "src/oracledb/impl/thin/connection.pyx", line 318, in oracledb.thin_impl.ThinConnImpl._connect_with_description File "src/oracledb/impl/thin/connection.pyx", line 291, in oracledb.thin_impl.ThinConnImpl._connect_with_address File "src/oracledb/impl/thin/protocol.pyx", line 317, in oracledb.thin_impl.Protocol._connect_phase_two File "src/oracledb/impl/thin/protocol.pyx", line 404, in oracledb.thin_impl.Protocol._process_message File "src/oracledb/impl/thin/protocol.pyx", line 382, in oracledb.thin_impl.Protocol._process_message File "src/oracledb/impl/thin/protocol.pyx", line 453, in oracledb.thin_impl.Protocol._receive_packet File "src/oracledb/impl/thin/packet.pyx", line 705, in oracledb.thin_impl.ReadBuffer.wait_for_packets_sync File "src/oracledb/impl/thin/transport.pyx", line 314, in oracledb.thin_impl.Transport.read_packet File "/home/rancid/.local/lib/python3.9/site-packages/oracledb/errors.py", line 181, in _raise_err raise error.exc_type(error) from cause oracledb.exceptions.DatabaseError: DPY-4011: the database or network closed the connection [Errno 104] Connection reset by peer Help: https://python-oracledb.readthedocs.io/en/latest/user_guide/troubleshooting.html#dpy-4011

But if in the same server i create a simple container (ubuntu or python3) and intall only the oracledb and execute the same code works perfectly.

I do not know what is the difference between server and container.

The problem occurs in any user of the server, from other servers in the same network i do not have the problem, it is only in this server (but not inside a docker in the same server).

How can i investigate the problem ?

Any tip of the problem ??

anthony-tuininga commented 3 months ago

What version of python-oracledb are you using? It looks like the server is dropping the connection on you based on the error: ConnectionResetError: [Errno 104] Connection reset by peer. If I know the version I can find out where exactly in the protocol the error occurs -- which may be helpful diagnosing the issue for you.

i-m-d commented 3 months ago

in server:

python = 3.9.2 oracledb==2.1.0

In docker container:

python = 3.10.12 oracledb==2.1.1

i-m-d commented 3 months ago

I have just updated the oracledb in server:

python = 3.9.2 oracledb==2.1.1

the same error:

In [1]: import oracledb ...: connection = oracledb.connect( ...: user=myuser, ...: password=mypass, ...: dsn=mydsn)


ConnectionResetError Traceback (most recent call last) File src/oracledb/impl/thin/transport.pyx:321, in oracledb.thin_impl.Transport.read_packet()

ConnectionResetError: [Errno 104] Connection reset by peer

The above exception was the direct cause of the following exception:

DatabaseError Traceback (most recent call last) Input In [1], in <cell line: 2>() 1 import oracledb ----> 2 connection = oracledb.connect( 3 user=myuser 4 password=mypass, 5 dsn=mydsn)

File ~/.local/lib/python3.9/site-packages/oracledb/connection.py:1158, in _connection_factory..connect(dsn, pool, conn_class, params, kwargs) 1156 message = "pool must be an instance of oracledb.ConnectionPool" 1157 raise TypeError(message) -> 1158 return conn_class(dsn=dsn, pool=pool, params=params, kwargs)

File ~/.local/lib/python3.9/site-packages/oracledb/connection.py:541, in Connection.init(self, dsn, pool, params, **kwargs) 539 else: 540 impl = thin_impl.ThinConnImpl(dsn, params_impl) --> 541 impl.connect(params_impl) 542 else: 543 impl = thick_impl.ThickConnImpl(dsn, params_impl)

File src/oracledb/impl/thin/connection.pyx:381, in oracledb.thin_impl.ThinConnImpl.connect()

File src/oracledb/impl/thin/protocol.pyx:377, in oracledb.thin_impl.Protocol._process_message()

File src/oracledb/impl/thin/connection.pyx:337, in oracledb.thin_impl.ThinConnImpl._connect_with_params()

File src/oracledb/impl/thin/connection.pyx:318, in oracledb.thin_impl.ThinConnImpl._connect_with_description()

File src/oracledb/impl/thin/connection.pyx:291, in oracledb.thin_impl.ThinConnImpl._connect_with_address()

File src/oracledb/impl/thin/protocol.pyx:312, in oracledb.thin_impl.Protocol._connect_phase_two()

File src/oracledb/impl/thin/protocol.pyx:399, in oracledb.thin_impl.Protocol._process_message()

File src/oracledb/impl/thin/protocol.pyx:377, in oracledb.thin_impl.Protocol._process_message()

File src/oracledb/impl/thin/protocol.pyx:448, in oracledb.thin_impl.Protocol._receive_packet()

File src/oracledb/impl/thin/packet.pyx:710, in oracledb.thin_impl.ReadBuffer.wait_for_packets_sync()

File src/oracledb/impl/thin/transport.pyx:323, in oracledb.thin_impl.Transport.read_packet()

File ~/.local/lib/python3.9/site-packages/oracledb/errors.py:181, in _raise_err(error_num, context_error_message, cause, **args) 179 message = f"{message}\n{context_error_message}" 180 error = _Error(message) --> 181 raise error.exc_type(error) from cause

DatabaseError: DPY-4011: the database or network closed the connection [Errno 104] Connection reset by peer Help: https://python-oracledb.readthedocs.io/en/latest/user_guide/troubleshooting.html#dpy-4011

anthony-tuininga commented 3 months ago

Interesting! It might be useful to provide the output when setting the environment variable PYO_DEBUG_PACKETS to any value before running your script in both places. In the one case it will show you the full output and in the other case you should only see the first few packets before the server disconnects. It would be good to compare those first few packets to see if there is any difference between them (there shouldn't be). The error shows that it is failing in the "data types" packet. One thought: can you add the "disable_oob" parameter to your connect() call? Can you also indicate what version of the database you are using?

i-m-d commented 3 months ago

Thanks Anthony for you help.

('Oracle Database 19c Enterprise Edition ', '19.0.0.0.0', '19.3.0.0.0', 'Production')

the difference using export PYO_DEBUG_PACKETS=1 in both server and docker are minimal, besides tipically variables: HOST , CONNECTION_ID, USER, AUTH_SESSKEY, AUH_PASSWORD change some (very little) HEX values, the same with disable_oob=True in the connection.

Server secuence (ommiting hex values): 2024-03-23 08:27:56.893 Sending packet [op 1] on socket 12 2024-03-23 08:27:56.894 Receiving packet [op 2] on socket 12 2024-03-23 08:27:56.894 Receiving packet [op 3] on socket 12 2024-03-23 08:27:56.896 Sending packet [op 4] on socket 13 2024-03-23 08:27:56.896 Sending packet [op 5] on socket 13 2024-03-23 08:27:56.915 Receiving packet [op 6] on socket 13 2024-03-23 08:27:56.915 Sending packet [op 7] on socket 13 2024-03-23 08:27:56.915 Sending packet [op 8] on socket 13 2024-03-23 08:27:56.915 Receiving packet [op 9] on socket 13 2024-03-23 08:27:56.915 Sending out of band break [op 10] on socket 13 2024-03-23 08:27:56.916 Sending packet [op 11] on socket 13 2024-03-23 08:27:56.916 Sending packet [op 12] on socket 13 2024-03-23 08:27:56.916 Receiving packet [op 13] on socket 13 2024-03-23 08:27:56.916 Sending packet [op 14] on socket 13 2024-03-23 08:28:56.925 Disconnecting transport [op 15] on socket 13

Docker secuence (ommiting hex values): 2024-03-23 07:35:31.578 Sending packet [op 1] on socket 5 2024-03-23 07:35:31.579 Receiving packet [op 2] on socket 5 2024-03-23 07:35:31.579 Receiving packet [op 3] on socket 5 2024-03-23 07:35:31.579 Sending packet [op 4] on socket 6 2024-03-23 07:35:31.580 Sending packet [op 5] on socket 6 2024-03-23 07:35:31.598 Receiving packet [op 6] on socket 6 2024-03-23 07:35:31.598 Sending packet [op 7] on socket 6 2024-03-23 07:35:31.598 Sending packet [op 8] on socket 6 2024-03-23 07:35:31.598 Receiving packet [op 9] on socket 6 2024-03-23 07:35:31.598 Sending out of band break [op 10] on socket 6 2024-03-23 07:35:31.598 Sending packet [op 11] on socket 6 2024-03-23 07:35:31.599 Sending packet [op 12] on socket 6 2024-03-23 07:35:31.599 Receiving packet [op 13] on socket 6 2024-03-23 07:35:31.599 Sending packet [op 14] on socket 6 2024-03-23 07:35:31.602 Receiving packet [op 15] on socket 6 2024-03-23 07:35:31.604 Sending packet [op 16] on socket 6 2024-03-23 07:35:31.680 Receiving packet [op 17] on socket 6 2024-03-23 07:35:31.687 Sending packet [op 18] on socket 6 2024-03-23 07:35:31.715 Receiving packet [op 19] on socket 6

anthony-tuininga commented 3 months ago

Thanks. Can you try using the disable_oob parameter when connecting to see if that resolves the issue for you? If it doesn't, can you e-mail me the complete packet output?

i-m-d commented 3 months ago

Hello Anthony,

i already tried it. the same with disable_oob=True in the connection.

i-m-d commented 3 months ago

I have just remembered that this server had the cx_oracle python module and instantcliet, but was removed, may be not completely ? some tips to check if some "garbage" is the problem ? Evidently the container is clean, as it is created from image and installed all from scratch.

i-m-d commented 3 months ago

I do not found your email to send the output. The output has "sensible" info from the company i work for it is possible to hide that info ?

anthony-tuininga commented 3 months ago

You can find my e-mail address on PyPI. You can hide the info if you would like, but that may also hide what is needed to diagnose the issue -- I leave that to your discretion!

i-m-d commented 3 months ago

Hello Anthony,

Email sent with three outputs:

1) FROM SERVER WITHOUT disable_oob (DO NOT WORK) 2) FROM SERVER WITH disable_oob=True (DO NOT WORK) 3) FROM DOCKER IN THE SAME SERVER (This works and execute sql)

Thank you for your help.

anthony-tuininga commented 2 months ago

Any progress on this based on the reply to the e-mail you sent?

i-m-d commented 2 months ago

Hello Anthony,

Magically now wirks correctly inside container and outside, really i do not know how was fixed, maybe an upgrade of an old python module, i did not upgrade all modules only 2 o 3.

I think this mistery will remain unresolved :(

Thank you very much for you time.