Incorrect data in clickhouse after migration

[VolhaRS]

Hello,

use clickhouse-mysql for migrate data from mysql to clickhouse. After migration some fields have incorrect value in clickhouse, if in mysql they have utt8 with russian symbols. But new data replicated correctly! If the same data migrate with flag --migrate-table, i see question marks instead russian text.

Step to reproduce. In mysql:

1. create database test;
2. CREATE TABLE test.books ( id int(11) NOT NULL AUTO_INCREMENT, author varchar(255) NOT NULL, book varchar(255) NOT NULL, PRIMARY KEY (id)) ENGINE=InnoDB CHARSET=utf8;
3. insert into test.books value('', 'Лукьяненко', 'Ночной дозор'),('', 'Стругацкие', 'Понедельник начинается в субботу'),('', 'Lukyanenko', 'Night watch'),('', 'Strugatsky', 'Monday starts on Saturday');

In clickhouse:

4. create database test
5. CREATE TABLE IF NOT EXISTS test.books ( primary_date_field Date default today(), id Int32, author String, book String) ENGINE = MergeTree(primary_date_field, (id), 8192);

Start migration from mysql to clickhouse:

6. clickhouse-mysql --src-host=mysql --src-tables=test.books --src-user=user --src-password=password --dst-host=clickhouse-server --dst-schema=test --dst-table=books --dst-user=user --dst-password=password --migrate-table

7. select data from mysql:

8. select data from clickhouse:

[sunsingerus]

Hello, have tested this issue, and have the following results:

1. Provided SQL INSERT statement does not work for me
2. Migration completed correctly. Options used:

   --src-server-id=1 \
   --nice-pause=1 \
   --log-level=debug \
   --src-host=127.0.0.1 \
   --src-user=reader \
   --src-password=qwerty \
   --src-tables=test.books \
   --dst-host=127.0.0.1 \
   --dst-schema=test \
   --dst-table=books \
   --csvpool \
   --csvpool-file-path-prefix=qwe_ \
   --mempool-max-flush-interval=60 \
   --mempool-max-events-num=10000 \
   --pump-data \
   --migrate-table

   Result:

Looks like it may be version-dependent or whatever else.

[sunsingerus]

Try to install new version from pypi (in case you are not using sources from master branch) and check again, if possible