apache / solr-operator

Official Kubernetes operator for Apache Solr
https://solr.apache.org/operator
Apache License 2.0
243 stars 112 forks source link

unable to take solr backup using s3 #597

Closed kannanvr closed 8 months ago

kannanvr commented 11 months ago

Hi Team,

Trying to take Solr backup to s3. Getting below error.

image

some directory are created in s3, but no data in it.

s3://solr/solrbkp-country/country/
[2023-08-02 22:06:29 IST]     0B index/77b5e96f-c787-449d-86c3-d15a3dd96fa2
[2023-08-02 22:06:29 IST]     0B shard_backup_metadata/
[2023-08-02 22:06:29 IST]     0B zk_backup_0/

Kindly refer the solrCloud.spec for backuprepositories and solrBackup yaml (Also collection country was created and inserted few data as well).

  backupRepositories:
  - name: "s3-backup"
    s3:
      baseLocation: "/"
      bucket: "solr"
      endpoint: "http://x.x.x.x"
      region: "us-east-1"
      credentials:
        accessKeyIdSecret:
          name: s3-cred
          key: access_key
        secretAccessKeySecret:
          name: s3-cred
          key: secret_key
---
apiVersion: solr.apache.org/v1beta1
kind: SolrBackup
metadata:
  name: solrbkp
  namespace: solr-cluster-2
spec:
  repositoryName: "s3-backup"
  solrCloud: test1
  collections:
    - country
  recurrence: # Store one backup daily, and keep a week at a time.
    schedule: "@every 5m"
    maxSaved: 7
HoustonPutman commented 11 months ago

I see the following error "Parent directory doesn't exist". That's a very strange error, and I think it's an error in the S3-repository plugin (or the backupManager itself), not the Solr Operator...

Is this happen everytime you try to backup or did it happen just once?

kannanvr commented 11 months ago

Hi @HoustonPutman ,

We are getting this error every-time. As i mentioned, the directory structure is created in s3, but files are not copied.

HoustonPutman commented 8 months ago

Yes, please bring this up on the Solr users mailing list, as it's a Solr issue not a Solr Operator issue. Sorry for the delay.

As a guess, I think you are using an "S3 compatible" product that Solr doesn't actually support, such as Minio. If that's the case, I'm not sure there will be a solution.