influxdata / influxdb

Scalable datastore for metrics, events, and real-time analytics
https://influxdata.com
Apache License 2.0
28.95k stars 3.56k forks source link

InfluxDB 2.4.0 not able to take the backup properly for few buckets. #25120

Open ravi9717 opened 4 months ago

ravi9717 commented 4 months ago

Steps to reproduce: List the minimal actions needed to reproduce the behavior.

  1. create multiple orgs and one default org with different tokens and different bucket names
  2. create multiple buckets in each org and set different data retention policies for different buckets but the same across various orgs
  3. Now try to create the backup for different buckets across different orgs using the default org token

Expected behaviour: Influx backup -o org -b bucket -t token path should create the backup for the bucket for all buckets in all orgs

Actual behaviour: It creating the backup randomly for different buckets across different orgs

Environment info:

Config: Copy any non-default config values here or attach the full config as a gist or file.

Logs: this is the log of the bucket for which we able to create the backup image (1)

this is the logs of the bucket for which we are not able to create the backup, it just only shows downloading the metadata after that it stops. image (2)

ravi9717 commented 4 months ago

I have observed one more thing even if I pass the wrong bucket name in the command it doesn't give any error it simply shows downloading the metadata if other than token something else is wrong it's not showing in the command output.

davidby-influx commented 4 months ago

Can you send the exact commands to reproduce the problem, using only the influx CLI?

Start with influx setup

Then the commands to create the organizations and buckets, and finally the back-up command.

I will be attempting to reproduce on v2.7, not v2.4, so it might be worthwhile for you to upgrade, as well.

ravi9717 commented 4 months ago

I have created all the buckets from the InfluxDB GUI.

data retention for the bucket I configured from the CLI

influx backup -o org -t token -b bucket_name path

ravi9717 commented 4 months ago

I have observed one more thing even if I pass the wrong bucket name in the command it doesn't give any error it simply shows downloading the metadata if other than token something else is wrong it's not showing in the command output.

Please re-produce this issue as well

davidby-influx commented 4 months ago

@ravi9717 - If you can provide a CLI script to reproduce the problem, we can take a look at it. But without the exact bucket names, retention periods, data inserted, etc, we will be unable to experiment to see if we can reproduce the problem.

ravi9717 commented 3 months ago

what I have observed with my data is that this issue doesn't depend on the bucket name or the data type or data retention.

Because we are running the InfluxDB with multiple orgs and we have multiple buckets inside and issue doesn't have any relation with the Raw data bucket, KPI bucket, or data retention period. It's happening randomly for any bucket.