search

moiot / gravity

A Data Replication Center
Apache License 2.0
910 stars 173 forks source link

myql table scanner cannot stop if source db deletes data #11

Closed ming535 closed 5 years ago

ming535 commented 5 years ago

We should consider start a snapshot consistent read.

Ryan-Git commented 5 years ago

it would be too expensive for large tables, and make parallel fetch quite difficult, if not impossible.

I'd prefer a local comparison along with an optional remote one for non-numeric columns(e.g. select 'abc'>'abcd') instead.

Ryan-Git commented 5 years ago

Why it cannot stop? We'll finish the task if no rows returned from query. https://github.com/moiot/gravity/blob/bb844133dbb673daa40294606e72edde026ef751/pkg/inputs/mysqlbatch/mysql_table_scanner.go#L224