aio-libs / aioodbc

aioodbc - is a library for accessing a ODBC databases from the asyncio
Apache License 2.0
311 stars 60 forks source link

MySQL, error in my_thread_global_end #79

Open unreg opened 7 years ago

unreg commented 7 years ago

Hi! I have some problem. I use on linux system:

For any query I get an error message, not exception:

Error in my_thread_global_end(): 2 threads didn't exit

Query is executed correctly, only error message on console.

Code is very simple, from example, with pool and without pool:

dsn = f'Driver=myodbc;User={USER};Password={PASSWORD};Database={DB};'

async with aioodbc.create_pool(dsn=dsn, loop=loop) as pool:
    async with pool.acquire() as conn:
        async with conn.cursor() as cur:
            await cur.execute('SHOW TABLES;')
            val = await cur.fetchone()

Same code with just pyodbc runs without error message:

import pyodbc

cnxn = pyodbc.connect(dsn)
cursor = cnxn.cursor()
cursor.execute('SHOW TABLES;')
r = cursor.fetchall()

MySQL settings is default. Why can this error message?

jettify commented 7 years ago

I saw this logs, my best guess is that problem somewhere in mysql odbc library.

Code in your example is not equivalent, since async context managers also explicitly close cursor and connections in connection pool, when in your case connection and cursor destroyed with gc or exit of interpreter.

jettify commented 7 years ago

Same code used for postgresql and sqlite without any issue.

jettify commented 7 years ago

Found related MySQL issue https://bugs.mysql.com/bug.php?id=64466 , aioodbc may open connection in one thread and close it in other, MySQL driver does not like this, other odbc dbs work pretty fine this way.

unreg commented 7 years ago

my best guess is that problem somewhere in mysql odbc library.

may open connection in one thread and close it in other, MySQL driver does not like this

likely to...

jettify commented 7 years ago

Can confirm opening connection in one thread and closing it in other is root cause, not sure what I can do about this, since it is more ODBC MySQL driver issue, other dbs work as expected.

Possible solution is custom implementation of TheadPoolExecutor that can route blocking code depending to proper worker using threa_id information.

jettify commented 7 years ago

Probably this should not be an issue for MySQL >= 5.7.9 since mysql_thread_end [1] is no longer required to be called in [2].

[1] https://dev.mysql.com/doc/refman/5.7/en/mysql-thread-end.html [2] https://github.com/mysql/mysql-connector-odbc/blob/4f6159d8dde269d9f90ccb74d239f047ed5ad3a3/driver/handle.c#L368-L375