influxdata / influxdb

Scalable datastore for metrics, events, and real-time analytics
https://influxdata.com
Apache License 2.0
28.52k stars 3.53k forks source link

influxd restore fails for a database name containing periods #16753

Open SpComb opened 4 years ago

SpComb commented 4 years ago

If you have a database name containing periods, then it is possible to backup the database, but it is not possible to restore the backup.

Steps to reproduce:

Create a database with a . in the name, take a backup, and attempt to restore it

  1. CREATE DATABASE "hcloud-infra.telegraf"
  2. influxd backup -host infra-influxdb1:8088 -database hcloud-infra.telegraf hcloud-infra.telegraf
  3. influxd restore -online -host infra-influxdb1:8088 -db hcloud-infra.telegraf -newdb 'hcloud-infra.telegraf-restore' hcloud-infra.telegraf

Note that the backup shard files will contain an extra ., because the database name is included at the start of the filename:

bash-4.4$ influxd backup -host infra-influxdb1:8088 -database hcloud-infra.telegraf hcloud-infra.telegraf
2020/02/06 13:43:24 backing up metastore to hcloud-infra.telegraf/meta.00
2020/02/06 13:43:24 backing up db=hcloud-infra.telegraf
2020/02/06 13:43:24 backing up db=hcloud-infra.telegraf rp=default shard=359 to hcloud-infra.telegraf/hcloud-infra.telegraf.default.00359.00 since 0001-01-01T00:00:00Z
2020/02/06 13:43:25 backing up db=hcloud-infra.telegraf rp=default shard=362 to hcloud-infra.telegraf/hcloud-infra.telegraf.default.00362.00 since 0001-01-01T00:00:00Z
2020/02/06 13:43:25 backing up db=hcloud-infra.telegraf rp=default shard=365 to hcloud-infra.telegraf/hcloud-infra.telegraf.default.00365.00 since 0001-01-01T00:00:00Z

Expected behavior: The database shards should be restored to a new hcloud-infra.telegraf-restore database.

Actual behavior: The metastore snapshot is succesfully restored and the new hcloud-infra.telegraf-restore database is created, but the shard restore fails and crashes with an error:

bash-4.4$ influxd restore -online -host infra-influxdb1:8088 -db hcloud-infra.telegraf -newdb 'hcloud-infra.telegraf-restore' hcloud-infra.telegraf
2020/02/06 13:48:10 Using metastore snapshot: hcloud-infra.telegraf/meta.00
2020/02/06 13:48:11 Restoring live from backup hcloud-infra.telegraf/hcloud-infra.telegraf.*
2020/02/06 13:48:11 Skipping mis-named backup file: hcloud-infra.telegraf/hcloud-infra.telegraf.default.00359.00
2020/02/06 13:48:11 error updating shards: strconv.ParseUint: parsing "default": invalid syntax
restore: strconv.ParseUint: parsing "default": invalid syntax

Environment info:

SpComb commented 4 years ago

It's pretty clear from the influxd restore source code why this is happening, with strings.Split(filepath.Base(fn), "."): https://github.com/influxdata/influxdb/blob/ff383cdc0420217e3460dabe17db54f8557d95b6/cmd/influxd/restore/restore.go#L443-L451

Because the uploadShardsLegacy only happens for a single database at a time, this could perhaps be fixed by removing the cmd.sourceDatabase prefix from the filename, and then only split the remaining parts?

SpComb commented 4 years ago

There's probably a second bug in the error-handling here, the "Skipping mis-named backup file ..." case is missing a continue, so it's crashing hard on the strconv.ParseUint instead of skipping the backup file as intended.

rodrigorega commented 2 years ago

I have this problem too restoring Influx 1.8 databases with periods:

# /usr/bin/influxd restore -db telegraf_redacted01.ab.redacted02.com -datadir /var/lib/influxdb/data /mnt/backups/20220329-130428/data/telegraf_redacted01.ab.redacted02.com
2022/03/30 11:52:35 Restoring offline from backup /mnt/backups/20220329-130428/data/telegraf_redacted01.ab.redacted02.com/telegraf_redacted01.ab.redacted02.com.*
restore: backup tarfile name incorrect format

I need to restore a full backup into a new server, "-portable" also not works for me because is not backing up users. Is there any workaround?

Regards