search

Qovery / Replibyte

Seed your development database with real data ⚡️
https://www.replibyte.com
GNU General Public License v3.0
4.17k stars 129 forks source link

MySQL why use --skip-extended-insert? #117

Open moisesrodriguez opened 2 years ago

moisesrodriguez commented 2 years ago

Looking at how mysqldump is used, I noticed the flag --skip-extended-insert. Why write INSERT statements using one-row syntax that includes only one record in one statement, if it will be much much slower than having multiple records in one INSERT statement? Specially when importing a lot of data.

fabriceclementz commented 2 years ago

Hi @moisesrodriguez This flag was added so that the final dump is as close as possible to that of PostgreSQL, the reason being that it was easier to parse without major change in the dump parser for assuring a compatibility with MySQL. Indeed, this has a performance impact but this will be changed in a future release, the objective being to have the most efficient tool possible.

Contributions are welcome to improve the tool.

evoxmusic commented 2 years ago

Hey @moisesrodriguez (happy to see you here :)) It's related to this https://github.com/Qovery/Replibyte/issues/36 Once we'll fix it for PSQL, it would be easy to port that change to MySQL. As @fabriceclementz said, feel free to contribute if you have some time. I don't think it's a big deal, to be honest, just a matter of 2 to 3 hours of work (tests included).

fabriceclementz commented 2 years ago

Hi @evoxmusic This is the next issue I will try to tackle, As COPY for Postgres, This will considerably improve the performance of Replibyte, I let you know my progress.

evoxmusic commented 2 years ago

Thank you @fabriceclementz - it's definitely a must have improvement to increase restore performances. I am also looking how we can better bench and watch Replibyte performances