github / gh-ost

GitHub's Online Schema-migration Tool for MySQL
MIT License
12.43k stars 1.26k forks source link

Can we process binlog event and rowcopy parallel? #430

Open sumonsun opened 7 years ago

sumonsun commented 7 years ago

Review source code ,we find gh-ost process event by this rule that "We give higher priority to event processing, then secondary priority to rowcopy"

One question,this only think by performance,decrease original table's write load?

From design logic,can we process binlog event and rowcopy parallel,we have more test ,gh-ost ddl speed slower than pt,so we want to speed up.

Could you have any good idea?

shlomi-noach commented 7 years ago

The writes are serial by design, and this design simplifies the logic as well as prevents locks on the table. In my personal experience, concurrent writes to the same table do not have better throughput. Others may have their own experience and I don't know of a definitive benchmark.

The serial nature of applying rows + binlog events is unlikely to change.

As for comparing gh-ost with pt-online-schema-change, I am of course curious and interested. FWIW speed comparison was not a priority when designing gh-ost, but instead stability, safety and visibility.

I'm curious what flags you used. Chunk-size? dml-batch-size? other?