search

nikepan / clickhouse-bulk

Collects many small inserts to ClickHouse and send in big inserts
Apache License 2.0
474 stars 86 forks source link

How to guarantee data reliability? #30

Closed 2hangchen closed 4 years ago

2hangchen commented 4 years ago

The data will multi in memory, and if the service dead when the data is saved too much, the data will be lost

nikepan commented 4 years ago

You can set a short dispatch period. There are currently no ways to increase reliability.

nikepan commented 4 years ago

All tools are made for their tasks. Those who need reliability can afford to write for their tasks. This utility is needed more for collecting site statistics on python and php and can afford to lose a second of data.

2hangchen commented 4 years ago

All tools are made for their tasks. Those who need reliability can afford to write for their tasks. This utility is needed more for collecting site statistics on python and php and can afford to lose a second of data.

Yes, I think we have to find a way to minimize the damage