backup with truenas s3

[zasim87]

Hello everyone! I would appreciate any help. We use Barman - Backup and Recovery Manager for cnpg Trying to run backups with truenas s3:

backup: 
    barmanObjectStore:
      destinationPath: "s3://bucket/"
      endpointURL: https://truenas.dom.mycompany.com:9000/
      s3Credentials:
        accessKeyId:
          name: s3-creds
          key: ACCESS_KEY_ID
        secretAccessKey:
          name: s3-creds
          key: ACCESS_SECRET_KEY
    retentionPolicy: "30d"

got the following error:

Could not connect to the endpoint URL: \"https://s3.dom.amazonaws.com/bucket\

probably it happens because truenas returns Region': 'dom' I could manage it with

        region:
          name: s3-creds
          key: region

After the region is added, backups will work properly.

still looks like the URL combining is a bit strange:

\"https://s3.dom.amazonaws.com/bucket\

it takes region and puts it together with amazonaws.com

Is it expected behavior?

[mikewallace1979]

We do not test against TrueNAS so I can't confirm whether what you see when omitting the region is expected behaviour.

I tried checking the endpointURL and region requirements in the TrueNAS docs but the documentation around the S3 service doesn't appear to be very comprehensive. One thing I did notice is that the S3 service in TrueNAS is deprecated and the recommended approach is now to use the Minio plugin instead.

[zasim87]

We use Minio plugin in the TrueNAS. and I couldn't connect until I added a region to the s3Credentials settings. Also very confusing is the line "Could not connect to the endpoint URL: \"https://s3.dom.amazonaws.com/bucket\" in the log when I try to connect to https://truenas.dom.mycompany.com:9000/.

[mikewallace1979]

I don't know exactly what is going on but s3.amazonaws.com is the default endpoint used in the boto3 library when connecting to S3, and s3.<region>.amazonaws.com is the endpoint format for non-default regions. I suspect what happens is that when the region is not specified boto3 ends up retrying using the s3.<region>.amazonaws.com address (in this case s3.dom.amazonaws.com) and therefore that is what ends up in the final error message.

If you are able to run barman-cloud-backup directly you could increase the verbosity with the -vv argument which would give you more insight into how the endpoint is being resolved.