search

Altinity / clickhouse-mysql-data-reader

utility to read mysql data
MIT License
369 stars 94 forks source link

DELETE & UPDATE support #140

Closed jackpgao closed 4 years ago

jackpgao commented 5 years ago

As the newest version of ClickHouse has supported delete and update, will you support it too?

sunsingerus commented 5 years ago

Yes, the plans are to support DELETE and UPDATE operations, however, no clear implementation timeline so far

ed-fruty commented 5 years ago

Any update for this? https://clickhouse.yandex/docs/en/query_language/alter/#alter-mutations

sunsingerus commented 5 years ago

No actual progress on this, busy with more priority tasks

MxBird commented 4 years ago

any update for this issues for 1 year?

jackpgao commented 4 years ago

Since data modification is asynchronous, this solution should not be implemented.

MxBird commented 4 years ago

Since data modification is asynchronous, this solution should not be implemented.

能介绍下你们的解决方案 怎么解决数据更新跟删除的么？

jackpgao commented 4 years ago

没有解决。 ClickHouse的数据修改依靠的是后台的异步Merge，是不可控的，binlog通过过来，乱序执行后，数据就不一致了。

MxBird commented 4 years ago

没有解决。 ClickHouse的数据修改依靠的是后台的异步Merge，是不可控的，binlog通过过来，乱序执行后，数据就不一致了。

能给个聊天方式么 微信或者QQ 想请教下clickhouse的相关问题

ericchuawc commented 4 years ago

so if the data in mysql is updated or deleted, it won't be sync to clickhouse?

poweroftrue commented 3 years ago

Hello @jackpgao, Even if the data modification is asynchronous it can be very useful, we can include a warning that data update/delete will take some time or near real-time.

Even if update/delete works in the background and breaks consistency, that's already happening in other tools.

For most use cases without this tool (using data pipeline), it already takes time to get rows by updated_at field in case data changed, deletes takes much more, and sometimes a full re-index.