Closed bsramin closed 8 months ago
Thanks for reporting the bug / feature.
I was able to reproduce and fix it with this patch:
diff --git a/spec/triki_spec.cr b/spec/triki_spec.cr
index be237cd..3cfb9d5 100644
--- a/spec/triki_spec.cr
+++ b/spec/triki_spec.cr
@@ -264,7 +264,8 @@ describe Triki do
INSERT INTO `some_table` (`email`, `name`, `something`, `age`) VALUES ('bob@honk.com','bob', 'some\\'thin,ge())lse1', 25),('joe@joe.com','joe', 'somethingelse2', 54),('dontmurderme@direwolf.com','direwolf', 'somethingelse3', 44);
INSERT INTO `another_table` (`a`, `b`, `c`, `d`) VALUES (1,2,3,4), (5,6,7,8);
INSERT INTO `some_table_to_keep` (`a`, `b`, `c`, `d`) VALUES (1,2,3,4), (5,6,7,8);
- INSERT INTO `one_more_table` (`a`, `password`, `c`, `d,d`) VALUES ('hello','kjhjd^&dkjh', 'aawefjkafe', 'wadus'), ('hello1','kjhj!', 892938, 'tradus'), ('hello2','moose!!', NULL, NULL);
+ INSERT INTO `one_more_table` (`a`, `password`, `c`, `d,d`) VALUES ('hello','kjhjd^&dkjh', 'aawefjkafe', 'wadus'), ('hello1','kjhj!', 892938, 'tradus'),
+ ('hello2','moose!!', NULL, NULL);
INSERT INTO `an_ignored_table` (`col`, `col2`) VALUES ('hello','kjhjd^&dkjh'), ('hello1','kjhj!'), ('hello2','moose!!');
SQL
database_dump = IO::Memory.new(string)
diff --git a/src/triki/insert_statement_parser.cr b/src/triki/insert_statement_parser.cr
index 11e451e..d215c33 100644
--- a/src/triki/insert_statement_parser.cr
+++ b/src/triki/insert_statement_parser.cr
@@ -1,7 +1,7 @@
class Triki
module InsertStatementParser
def parse(obfuscator, config, input_io, output_io)
- input_io.each_line(chomp: false) do |line|
+ while line = input_io.gets(';')
if table_data = parse_insert_statement(line)
table_name = table_data["table_name"].as(String)
columns = table_data["column_names"].as(Array(String))
The difference is mysql client 5.7 & 8.2 exports:
INSERT INTO `employees` (`emp_no`, `birth_date`, `first_name`, `last_name`, `gender`, `hire_date`) VALUES (10001,'1953-09-02','Jason','Feil','M','1986-06-26'),(10002,'1964-06-02','Bezalel','Simmel','F','1985-11-21'),(10003,'1959-12-03','Parto','Bamford','M','1986-08-28'),(10004,'1954-05-01','Chirstian','Koblick','M','1986-12-01'),(10005,'1955-01-21','Kyoichi','Maliniak','M','1989-09-12'),(10006,'1953-04-20','Anneke','Preusig','F','1989-06-02'),(10007,'1957-05-23','Tzvetan','Zielinski','F','1989-02-10');
And MariaDB client 10.19 exports extended inserts as :
❯ mysqldump --version
mysqldump Ver 10.19 Distrib 10.11.6-MariaDB, for debian-linux-gnu (x86_64)
INSERT INTO `employees` (`emp_no`, `birth_date`, `first_name`, `last_name`, `gender`, `hire_date`) VALUES (10001,'1953-09-02','Jason','Feil','M','1986-06-26');
(10002,'1964-06-02','Bezalel','Simmel','F','1985-11-21'),
(10003,'1959-12-03','Parto','Bamford','M','1986-08-28'),
(10004,'1954-05-01','Chirstian','Koblick','M','1986-12-01'),
(10005,'1955-01-21','Kyoichi','Maliniak','M','1989-09-12'),
(10006,'1953-04-20','Anneke','Preusig','F','1989-06-02'),
(10007,'1957-05-23','Tzvetan','Zielinski','F','1989-02-10');
MariaDB change affecting MariaDB mysqldump
>= 10.7.1 https://github.com/MariaDB/server/commit/586d6a2520de23d8e5a3cc6ccfd29dbd728dba69
So now extended inserts are multi-line. Can you try with the patch?
@bsramin can you post your mysqldump --version
output?
HI, with RDS/Mysql 8, if a dump is created with multiple-row INSERT statements, break the dump file by writing the insert like this:
(example)
(wrong semicolon after first row and missing VALUES)