Closed GoogleCodeExporter closed 9 years ago
Please check if any other code block/api call deserves python thread-safe block.
Original comment by trebor74hr@gmail.com
on 1 Apr 2010 at 2:24
>>>>
We running some python code in
parallel - several threads, and each thread does access to DB2 database
using ibm_db adapter. This causes python.exe (win32) to crash - without
much information left behind - traceback is messed-up.
<<<<
can you give me a sample test case through which we can regenerate this
problem.
Because in general python threads acquires GIL before calling Python/C API
function.
So, above problem should not be happen.
>>>>
Looking into to C code of ibm_db.c I found no thread-safe blocks
Therefore I patched ibm_db.c with:
Py_BEGIN_ALLOW_THREADS;
...
Py_END_ALLOW_THREADS;
<<<<
By default Python handles Thread safe property for Python/C API function call,
Through acquiring GIL.
And by using “Py_BEGIN_ALLOW_THREADS” we are releasing GIL before calling
db2 sql api call, so through instead of gaining thread safe we are allowing
other
thread to do their job.
But through this we will get performance gain.
Original comment by rahul.pr...@in.ibm.com
on 10 May 2010 at 8:25
I'm sorry but I can't give you test case for reproducing the problem, just short
explanation:
on some moment a thread is created which does some db
select/delete/update/insert,
and in the "same" moment main program continues - and the logic can also do some
select/upd/del/ins. From time to time this crashes. After commenting out thread
part,
it is ok, putting thread part alive again, it failed. After putting Py_*THREAD
macro
around some "critical" db2 calls it stopped failing.
Original comment by trebor74hr@gmail.com
on 10 May 2010 at 3:07
I tried a lot to reproduce this problem with no luck.
Further by adding Py_*THREAD macro you are making those db calls as non
interrupting
calls(which can't interrupt other threads). so, during these db calls other
threads
will be able to do their works.
Are you using same connection object for both children and main thread?
Original comment by rahul.pr...@in.ibm.com
on 12 May 2010 at 8:24
to be more specific, this is done in django application, where request
middleware
spawns new thread which does some syncing job, and then mw passes control to the
django view which can modify data in the same db. I don't know if this is the
same
connection, don't know details how django does this (only django ORM api is
used). My
assumption is: yes, they have same connection.
Original comment by trebor74hr@gmail.com
on 17 May 2010 at 1:14
I have created a patch for unblocking costly db calls(now these calls will not
block
other threads execution).
please find the patch in attachment and test it in your environment.
After using this patch let me know how it works and this is able to resolve your
problem or not.
Depending upon your feedback i shall include this patch in our next release.
Original comment by rahul.pr...@in.ibm.com
on 20 May 2010 at 10:43
Attachments:
Sorry for delay, currently I'm not able to test this, but will next week. Did
overview, noticed that you did a lot "ALLOW_THREADS" wraps, looks good, and
hope this
won't decrease performance.
Original comment by trebor74hr@gmail.com
on 3 Jun 2010 at 11:03
Finally I managed to test the patched version, and good news is that problems I
had before (python.exe crashes) are not present any more. I think you could
include this patch in next release.
Original comment by trebor74hr@gmail.com
on 29 Jun 2010 at 10:44
Thanks robert for testing this patch. we will include this patch to our next
release.
Original comment by rahul.pr...@in.ibm.com
on 2 Jul 2010 at 6:40
Released with ibm_db-1.0.3
Original comment by rahul.pr...@in.ibm.com
on 9 Aug 2010 at 7:45
Original issue reported on code.google.com by
trebor74hr@gmail.com
on 1 Apr 2010 at 2:22Attachments: