SPOOL_ATTRVAL_FORCEGET segfault above 64 busy/opened connections

[markfinn]

Hi,

On 6.1.0 I can cause a segfault with:

cx_Oracle.SessionPool(user, pw, dsn, 1, 4, 1, threaded=True, getmode=cx_Oracle.SPOOL_ATTRVAL_FORCEGET)

then calling enough pool.acquire() to get over 64 connections

[anthony-tuininga]

Can you provide a bit more information? Perhaps the entire script that is causing the problems? I just tried it on Linux, 12.2 client and database and created 150 connections without any difficulty.

[markfinn]

11xe database, 12.2 client, 64 bit linux.

Disentangling a simple test case from my code will take me some time, I was hoping it would show up easily.

I think what is happening in my very convoluted code is that I am acquiring a connection, performing a query and fetch, but before they are done, acquiring again 64 times. so there are 64 connections with queries and fetches in flight.

If that's roughly what you tried I will work on paring my code down to a reliable test case.

[anthony-tuininga]

This is my test case. I tried it with 11.2 and 12.2 databases. I did not have any problems. Can you try this yourself? Is there a hard limit on the number of connections that can be created perhaps? Perhaps it is failing when the limit is exceeded? Can you give this code a try and let me know if it fails for you, too?

import cx_Oracle
import threading

NUM_ITERS = 80

pool = cx_Oracle.SessionPool("cx_Oracle", "welcome", DSN,
        min=1, max=4, increment=1, threaded=True,
        getmode=cx_Oracle.SPOOL_ATTRVAL_FORCEGET)

def ThreadRun(iterNum, conn):
    cursor = conn.cursor()
    cursor.execute("select count(*) from TestNumbers")
    count, = cursor.fetchone()
    print("Iter", iterNum, "count is", count)

threads = []
for i in range(NUM_ITERS):
    conn = pool.acquire()
    thread = threading.Thread(target=ThreadRun, args=(i + 1, conn))
    threads.append(thread)
for thread in threads:
    thread.start()
for thread in threads:
    thread.join()
print("Done!")

[markfinn]

That does blow up for me. Note that my database doesn't have a TestNumbers table. I hadn't gotten around to adding it yet and found a failure first.

I added print pool.busy, pool.opened right after conn = pool.acquire()

and get:

(venv) 14:46:56-mark/tmp$ python test.py 1 1 2 2 3 3 4 4 5 5 6 6

...

56 56 57 57 58 58 59 59 Segmentation fault (core dumped) (venv) 14:46:58-mark/tmp$

Here's the backtrace:

0x00007ffff0b193b9 in kpuspsessionget () from /home/mark/data/oracle_experiment/clean_cx_oracle/venv/oracle/instantclient_12_2/libclntsh.so.12.1 0x00007ffff093cb4c in OCISessionGet () from /home/mark/data/oracle_experiment/clean_cx_oracle/venv/oracle/instantclient_12_2/libclntsh.so.12.1 0x00007ffff33e2f62 in dpiOcisessionGet (envHandle=0xbd7080, handle=0x18b4c50, authInfo=0xc4d0d8, connectString=, connectStringLength=, tag=, tagLength=0, outTag=0x7fffffffd568, outTagLength=0x7fffffffd570, found=0x7fffffffd574, mode=1, error=0x7fffffffd300) at odpi/src/dpiOci.c:2344 0x00007ffff33fc47b in dpiConngetSession (error=0x7fffffffd300, authInfo=0xc4d0d8, params=0x7fffffffd510, connectStringLength=29, connectString=0xc306f0 "OCI:SP:XoVZ0SefCkrgUwEBAH8vOC", mode=1, conn=0x18b4c30) at odpi/src/dpiConn.c:462 dpiConnget (conn=0x18b4c30, userName=userName@entry=0x0, userNameLength=userNameLength@entry=0, password=password@entry=0x0, passwordLength=passwordLength@entry=0, connectString=0xc306f0 "OCI:SP:XoVZ0SefCkrgUwEBAH8vOC", connectStringLength=29, createParams=0x7fffffffd510, pool=0xbd6ef0, error=0x7fffffffd300) at odpi/src/dpiConn.c:317 0x00007ffff33fc830 in dpiPoolacquireConnection (pool=0xbd6ef0, userName=userName@entry=0x0, userNameLength=userNameLength@entry=0, password=password@entry=0x0, passwordLength=passwordLength@entry=0, params=params@entry=0x7fffffffd510, conn=0x7ffff336ef20, error=0x7fffffffd300) at odpi/src/dpiPool.c:35 0x00007ffff33ff02e in dpiConn_create (context=0xba8df0, userName=0x0, userNameLength=0, password=0x0, passwordLength=0, connectString=0x0, connectStringLength=0, commonParams=0x7fffffffd4d0, createParams=0x7fffffffd510, conn=0x7ffff336ef20) at odpi/src/dpiConn.c:1126 0x00007ffff33ffef6 in Connection_Init (self=0x7ffff336ef10, args=, keywordArgs=) at src/Connection.c:854 0x000000000047f18a in type_call (type=, args=0x7ffff7f93050, kwds=0x7ffff335f7f8) at Objects/typeobject.c:765 0x000000000042315a in PyObject_Call (func=0x7ffff3625960 , arg=arg@entry=0x7ffff7f93050, kw=kw@entry=0x7ffff335f7f8) at Objects/abstract.c:2547 0 0x00007ffff33d94ff in SessionPool_Acquire (self=0x7ffff33a4b20, args=0x7ffff7f93050, keywordArgs=) at src/SessionPool.c:367 0x00000000004ba39a in call_function (oparg=, pp_stack=0x7fffffffd880) at Python/ceval.c:4352 PyEval_EvalFrameEx (f=f@entry=0x888a00, throwflag=throwflag@entry=0) at Python/ceval.c:2989 0x00000000004bacb8 in PyEval_EvalCodeEx (co=co@entry=0x7ffff7f7dcb0, globals=globals@entry=0x7ffff7f68168, locals=locals@entry=0x7ffff7f68168, args=args@entry=0x0, argcount=argcount@entry=0, kws=kws@entry=0x0, kwcount=kwcount@entry=0, defs=defs@entry=0x0, defcount=defcount@entry=0, closure=closure@entry=0x0) at Python/ceval.c:3584 0x00000000004bade2 in PyEval_EvalCode (co=co@entry=0x7ffff7f7dcb0, globals=globals@entry=0x7ffff7f68168, locals=locals@entry=0x7ffff7f68168) at Python/ceval.c:669 0x00000000004eaa7d in run_mod (arena=0x8370c0, flags=0x7fffffffdb80, locals=0x7ffff7f68168, globals=0x7ffff7f68168, filename=0x7fffffffe12e "test.py", mod=0x887b30) at Python/pythonrun.c:1376 PyRun_FileExFlags (fp=0x86b690, filename=0x7fffffffe12e "test.py", start=, globals=0x7ffff7f68168, locals=0x7ffff7f68168, closeit=1, flags=0x7fffffffdb80) at Python/pythonrun.c:1362 0x00000000004ec2f1 in PyRun_SimpleFileExFlags (fp=0x86b690, filename=0x7fffffffe12e "test.py", closeit=1, flags=0x7fffffffdb80) at Python/pythonrun.c:948 0x0000000000415e71 in Py_Main (argc=, argv=) at Modules/main.c:640 0x00007ffff70991c1 in __libc_start_main (main=0x415030

, argc=2, argv=0x7fffffffdd38, init=, fini=, rtld_fini=, stack_end=0x7fffffffdd28) at ../csu/libc-start.c:308 0x000000000041505e in _start ()

[anthony-tuininga]

Interesting. Can you try with standalone connections instead of using a session pool? Can you try with other versions of the client? Including the 11g XE "client" that is part of the database installation?

[markfinn]

using a regular connect instead of the pool returns cx_Oracle.DatabaseError: ORA-12519: TNS:no appropriate service handler found on connection 60, which seems to be the one that would have segfaulted the pool.

[markfinn]

trying the libraries from the 11xe install made no difference with either the pool or connect() case

[cjbj]

As a possible workaround, try bumping the DB 'processes' parameter. See 'Configuring the Database For Testing' on page 317 of http://www.oracle.com/technetwork/topics/php/underground-php-oracle-manual-098250.html

[anthony-tuininga]

I can replicate this issue with 12.2 client and database. Use the workaround suggested by @cjbj for now or include a limit in your own code to avoid it.

[anthony-tuininga]

This is being tracked internally with bug 27170191.

[markfinn]

Any update on this? it was closed without resolution.

[anthony-tuininga]

Unfortunately, there is no update. It is still sitting in the queue. If you have a support contract with Oracle I'd suggest specifically referencing the bug number noted above!