search

TonicAI / condenser

Condenser is a database subsetting tool
https://www.tonic.ai
MIT License
312 stars 48 forks source link

Use separate connection for writes where db_helper.copy_rows called... #15

Closed wankata closed 4 years ago

wankata commented 4 years ago

with the same source and destination connection.

With the MySQL database I am playing around, I get mysql.connector.errors.InternalError: Unread result found. The reason is, that db_helper.copy_rows writes while it is reading from the same connection. It is pretty tricky, because when you read the code of the method, you see copy_rows(source, destination), but once you call the method with the same connection, aka copy_rows(destination_conn, destination_conn) in subset.py

So I fixed the call to use separate connections for source and destination to avoid the annoying error.

Hope, it helps!

Ivan

wankata commented 4 years ago

Ignore this pull request, please. It create different set of problems. First, my downstream tables remain empty. After playing around with it, I struggled with 1412 (HY000): Table definition has changed

The only thing, that really worked for me is to make the cursor buffered. This is more or less a workaround that may hide a real problem, but as far as I can see the results for now, it works correctly.

I will make a new pull request with the buffered cursor. But one more time - this solution may hide another problem in the way we are working with the DB.

acolombi commented 4 years ago

Okay. I haven't had a chance to deeply consider this change quite yet. But I suppose that's for the best ;-) I'll hopefully get to it some time this week, to look at things carefully, and maybe provide a fix.