search

techouse / mysql-to-sqlite3

Transfer data from MySQL to SQLite
https://techouse.github.io/mysql-to-sqlite3/
MIT License
217 stars 31 forks source link

Fixed wrong slugify reference, cursor error, added MYSQL port #1

Closed jcjveraa closed 5 years ago

jcjveraa commented 5 years ago

Thanks for making this πŸ‘ It took me a short while to actually get it working due to some issues which I fixed in this commit.

The original requirements.txt pointed to slugify = https://github.com/zacharyvoase/slugify which is probably not the version that we should use, as it throws errors. Updated it to python-slugify, https://github.com/un33k/python-slugify.

I got an error InternalError (Unread result found) when using the code as-is, and this was fixed by setting the mysql connection to buffered (I did not heavily troubleshoot this, just applied the fix and it worked).

My server was not running on port 3306 (but on 3307 - running MariaDB 5 and MariaDB 10 in parallel), so added the option to specify the port.

techouse commented 5 years ago

Great work! Thanx!

techouse commented 5 years ago

However a buffered cursor might not be what you'd want on a large database because it has to load all the data into memory before processing. I tested the original code on a 20GB MySQL 8 database and it worked just fine.

jcjveraa commented 5 years ago

Perhaps we can set it False per default and add it as an additional command line parameter - - use-buffered-cursors with the help-hint 'in case you encounter the InternalError (Unread result found) error, you can try to use this option'?

Op ma 25 mrt. 2019 10:16 schreef Klemen TuΕ‘ar notifications@github.com:

However a buffered cursor might not be what you'd want on a large database because it has to load all the data into memory before processing. I tested the original code on a 20GB MySQL 8 database and it worked just fine.

β€” You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/techouse/mysql-to-sqlite3/pull/1#issuecomment-476112860, or mute the thread https://github.com/notifications/unsubscribe-auth/ADwnndxtfIeVz8f3EhLakHumy2YchAH_ks5vaJPngaJpZM4cFAwT .

techouse commented 5 years ago

I was thinking the same thing. πŸ˜ƒ