search

go-mysql-org / go-mysql-elasticsearch

Sync MySQL data into elasticsearch
MIT License
4.12k stars 800 forks source link

Table not synced when there is a generated column #213

Open huygn opened 6 years ago

huygn commented 6 years ago

My contacts table has been synced until I run this sql to add a generated column

ALTER TABLE contacts
    ADD full_name VARCHAR(512) GENERATED ALWAYS AS (CONCAT(first_name,' ',last_name)) STORED NOT NULL;

When I delete the index and run go-mysql-elasticsearch again (deleted var dir), contacts table didn't get synced at all - it somehow got skipped in the dump/parse process (previously took ~12sec to dump my db, after I ran the query it took 0.4sec), other tables are still synced though.

siddontang commented 6 years ago

@huygn

Now go-mysql-elasticsearch doesn't support DDL, you have to delete the master file and re-sync all. As you see above, it will first use mysqldump to dump all the data which may take a long time.

I will try to support simple DDL like adding/removing columns later.

huygn commented 6 years ago

@siddontang

you have to delete the master file and re-sync all

Did you mean the master file inside var dir? I did rm -rf ./var && ./bin/go-mysql-elasticsearch but then it skip the whole table

huygn commented 6 years ago

I notice this problem only affect when you have a generated column in a table, if I do like below the table got synched again

ADD full_name VARCHAR(512)
siddontang commented 6 years ago

oh, generated column is not in the future plan.