search

motleyagency / influxdb-incremental-restore

InfluxDB incremental restore script
MIT License
14 stars 0 forks source link

Unable to restore incremental backups #13

Open ajayanaj opened 5 years ago

ajayanaj commented 5 years ago

Hi,

We just found this utility is useful and tried to restore our influxdb incremental backups, but we are getting issues as below.

1) Initially the restoration failed with the below error, ERR: partial write: field type conflict: input field "value" on measurement "measurement_name" is type float, already exists as type integer dropped=77 So as suggested by you I tried "-useTargetMeasurements" 2) Now I'm getting the below error Using target measurements: host::tag,metric_type::tag,value::float Using target measurements: host::tag,metric_type::tag,value::float ERROR: Command failed: influx -host 127.0.0.1 -port 8086 -format=column -execute SELECT host::tag,metric_type::tag,value::float INTO test_db..agents.online FROM test_db_20190630T000016Z..agents.online GROUP BY * error parsing query: too many segments in "hip_restore_test".."agents".online at line 1, char 1

Also we noticed that the field types are changing to "float" in restored database (For some fields). In original database field type for some measurements is "Integer" but in incremental_restored backup database the field type is "float"

Can you please check and let us know if any workarounds are there?

petetnt commented 5 years ago

Hi @ajayanaj,

sadly we haven't run into such issue after implementing the useTargetMeasurements flag ourselves. Could you try running the utility against a smaller subset of incremental backups? If that doesn't work, try to rename the backup prefixes to something else too (eg. the files in https://github.com/motleyagency/influxdb-incremental-restore#folder-structure to something like 1.meta, 1.s1.tar.gz and so on)

ajayanaj commented 5 years ago

@petetnt Thank you for your suggestions, but unfortunately the suggested workarounds not working for us. We tried with 5 hour data, 1 hour data respectively to restore, and we ended up with errors.

The second suggestion is bit complicated, while renaming the folder we need to change the respective gz file name in manifest file also.

I think the restoration script would work if the field types are not changing while restoring the backups. Any particular reason for the field type change?