Shard restoration is painfully slow

[egeexyz]

I was looking for a solution to keep two InfluxDB instances in sync when I found this repo. I tried restoring a backup from our Influx01 to Influx02 running on a t2.medium and it took 15 minutes for a modestly sized dataset.

Also, the merge attempt failed several times before completing.

Any ideas on what is causing this issue? Is merging the data that resource intensive?

[petetnt]

Hi @egee-irl,

yep, we are suffering from the same issue. The merging is resource intensive AND InfluxDB (endpoint and the db) chokes rather easily when doing the INSERT INTO clause.

Some things you could try:

• Increase the concurrency with the -concurrency
• If your machine is resource limited, try to change the values of -pps flag to limit chocking the DB
• Disable query-timeout: this should help with the merge attempts
• Set a CPU limit to your instance to limit (again) chocking the machine

There's also more insights on the InfluxDB issue from another user: https://github.com/influxdata/influxdb/issues/9593#issuecomment-416812425