blue-yonder / turbodbc

Turbodbc is a Python module to access relational databases via the Open Database Connectivity (ODBC) interface. The module complies with the Python Database API Specification 2.0.
http://turbodbc.readthedocs.io/en/latest
MIT License
607 stars 85 forks source link

executemany does not execute sql statement #403

Open Takken0 opened 6 months ago

Takken0 commented 6 months ago

Hello, I have been testing out turbodbc on a mariadb datebase and its been working out well up until now. For some reason, executemany wont execute sql. It wont spit out an error, it wont freeze or anything at all. The code will execute as if it has executed just fine, but when I check to table to see if it executed, nothing on the table will change at all. I wasn't sure if this was a database issue or a table issue, so I replaced the connector from turbodbc with the connector for pypyodbc. When I run that, it'll work just fine and input the code with executemany. So if this is a turbodbc issue, what is causing issues and what can I do to fix it. Strangely enough, when running executemany with only one tuple set, it will insert it into the table without any issues, but when I do two or more, it will still act just fine, but will not actually insert the data into the table. I am running turbodbc on python-pypy3.9 and on the latest version. of turbodbc. If it has to do with my code, that would also be great to know.

def test(): from turbodbc import connect connection = connect(dsn='My-Test-Server') cursor = connection.cursor() cursor.fast_executemany = True sql = "INSERT INTO test_table (id, item1, item2) VALUES (?, ?, ?)" tuples=[(2, 'tst1', 'tst2'), (3, 'tst3', 'tst4')] cursor.executemany(sql, tuples) connection.commit() cursor.close() connection.close()

test()

pacman82 commented 6 months ago

Since Maria DB is one of the databases included in the test setup of odbc-api (the Rust ODBC bindings) I can confirm, that bulk inserts work well with that database.

For diagnosis, if I remember correctly turbodbc has features for logging diagnostics. Have you tried activating it. It is not unlikely that the statement gets executed overall, but for each row there is a warning of why it is not inserted.

Best, Markus

Takken0 commented 6 months ago

Since Maria DB is one of the databases included in the test setup of odbc-api (the Rust ODBC bindings) I can confirm, that bulk inserts work well with that database.

For diagnosis, if I remember correctly turbodbc has features for logging diagnostics. Have you tried activating it. It is not unlikely that the statement gets executed overall, but for each row there is a warning of why it is not inserted.

Best, Markus

Sorry that I took so long to respond. Tis the season after all. Anyway, I tried looking into the documentation and I culd'nt find anything about logging there. I couldn't find anything on logging when searching it up. So I assumed that you meant odbc logging, so I turned on trace file and reran the program, but no tracefile was made, so I'm at a loss. I'll try to see if theres any other odbc logging I can try, but some help in finding what you mean by logging features would be great. Thanks, and sorry I took so long to respond.

pacman82 commented 6 months ago

if I remember correctly

I did not remember correctly. Checked the code. There is nothing which would give you a hint of anything with a severity below error.

Sorry that I took so long to respond

Turns out, I should be sorry, for making you chase a red herring.

I would still hypothesize, that the statement actually executes successfully, but each individual row may fail to be inserted. This can have all kinds of reasons, but usually it is a violated constraint. This would be reported by ODBC as SUCCESS_WITH_INFO severity. To check you may want to try the same insert statements with the command line tools like odbcsv or odbc2parquet crank up the verbosity, and prepare just one or two lines of input data to witness the warnings.

Best, Markus