[Bug]: files_external with AmazonS3 does not find pre-existing buckets outside eu-west-1

⚠️ This issue respects the following points: ⚠️

[X] This is a bug, not a question or a configuration/webserver/proxy issue.
[X] This issue is not already reported on Github (I've searched it).
[X] Nextcloud Server is up to date. See Maintenance and Release Schedule for supported versions.
[X] Nextcloud Server is running on 64bit capable CPU, PHP and OS.
[X] I agree to follow Nextcloud's Code of Conduct.

Bug description

I have a pre-existing AWS S3 bucket in the us-east-1 region. Connecting it in the "files_external" admin settings by supplying the "bucket name" always fails. The error is a "409 conflict" - Nextcloud is trying to create a NEW bucket. It's also strangely trying to create the bucket in the "eu-west-1" region.

What might be happening:

"files_external" looks for the bucket in the eu-west-1 region
It doesn't find it, so it tries to create a new bucket
AWS rejects this bucket-create operation because S3 bucket names are globally unique. The bucket already exists in us-east-1, so a bucket with the same name cannot be created in eu-west-1

Related - I know if I let Nextcloud create a new bucket with a name that doesn't exist, it always creates the bucket in the eu-west-1 region.

Fix - The fix to this issue is to just set the region field (in my case us-east-1). But, between the weird error and the documentation making region sound optional for connecting buckets, it wasn't immediately obvious that I needed to set the region.

Steps to reproduce

In AWS, create a bucket in a region other than "eu-west-1" (and set up a key for Nextcloud access).
Navigate to the "external storage" admin setting. (/settings/admin/externalstorages)
Select "AmazonS3" as the type. Enter the bucket name and the key authentication info.
Click the button to verify the external storage configuration.
The verify will show an error from AWS with "409 conflict", that a "bucket create" operation in the "eu-west-1" region failed because the bucket name already exists.

Expected behavior

The "region" field is considered optional by Nextcloud documentation. If "region" is optional, then a user would expect Nextcloud to figure out which region a bucket is in by its globally unique bucket name.

If it's not practical to patch the "files_external" app to find which region a bucket is in, then an alternative solution would be to update the external storage AmazonS3 documentation to make it clear that the "region" field is required for pre-existing buckets: https://docs.nextcloud.com/server/latest/admin_manual/configuration_files/external_storage/amazons3.html

Installation method

Community Manual installation with Archive

Operating system

Debian/Ubuntu

PHP engine version

PHP 8.1

Web server

Nginx

Database engine version

MySQL

Is this bug present after an update or on a fresh install?

Fresh Nextcloud Server install

Are you using the Nextcloud Server Encryption module?

Encryption is Disabled

What user-backends are you using?

[X] Default user-backend (database)
[ ] LDAP/ Active Directory
[ ] SSO - SAML
[ ] Other

Configuration report

{
    "system": {
        "instanceid": "***REMOVED SENSITIVE VALUE***",
        "apps_paths": [
            {
                "path": "\/var\/www\/nextcloud\/apps",
                "url": "\/apps",
                "writable": true
            },
            {
                "path": "\/var\/www\/nextcloud\/extra-apps",
                "url": "\/extra-apps",
                "writable": true
            }
        ],
        "supportedDatabases": [
            "mysql"
        ],
        "log_type": "file",
        "logfilemode": 416,
        "logfile": "\/var\/www\/nextcloud\/nextcloud.log",
        "loglevel": 2,
        "logdateformat": "F d, Y H:i:s",
        "passwordsalt": "***REMOVED SENSITIVE VALUE***",
        "secret": "***REMOVED SENSITIVE VALUE***",
        "trusted_domains": [
            "localhost",
            "***REMOVED SENSITIVE VALUE***",
            "***REMOVED SENSITIVE VALUE***",
            "***REMOVED SENSITIVE VALUE***",
            "***REMOVED SENSITIVE VALUE***"
        ],
        "dbtype": "mysql",
        "version": "25.0.0.12",
        "overwrite.cli.url": "http:\/\/localhost",
        "dbname": "***REMOVED SENSITIVE VALUE***",
        "dbport": "",
        "dbtableprefix": "oc_",
        "mysql.utf8mb4": true,
        "dbuser": "***REMOVED SENSITIVE VALUE***",
        "dbpassword": "***REMOVED SENSITIVE VALUE***",
        "default_phone_region": "US",
        "installed": true,
        "mail_from_address": "***REMOVED SENSITIVE VALUE***",
        "mail_smtpmode": "smtp",
        "mail_sendmailmode": "smtp",
        "mail_domain": "***REMOVED SENSITIVE VALUE***",
        "mail_smtphost": "***REMOVED SENSITIVE VALUE***",
        "mail_smtpsecure": "tls",
        "mail_smtpauth": 1,
        "mail_smtpauthtype": "LOGIN",
        "mail_smtpname": "***REMOVED SENSITIVE VALUE***",
        "mail_smtppassword": "***REMOVED SENSITIVE VALUE***",
        "mail_smtpport": "587",
        "twofactor_enforced": "false",
        "twofactor_enforced_groups": [
            "admin"
        ],
        "twofactor_enforced_excluded_groups": [],
        "app_install_overwrite": [
            "files_trackdownloads",
            "files_ebookreader",
            "ocsms",
            "files_3d",
            "memories"
        ],
        "memories.exiftool": "\/var\/www\/nextcloud\/apps\/memories\/exiftool-bin\/exiftool-amd64-glibc",
        "maintenance": false,
        "theme": ""
    }
}

List of activated Apps

Enabled:
  - admin_audit: 1.15.0
  - camerarawpreviews: 0.8.0
  - cloud_federation_api: 1.8.0
  - comments: 1.15.0
  - contacts: 5.0.2
  - contactsinteraction: 1.6.0
  - dashboard: 7.5.0
  - dav: 1.24.0
  - drawio: 1.0.5
  - federatedfilesharing: 1.15.0
  - federation: 1.15.0
  - files: 1.20.0
  - files_external: 1.17.0
  - files_sharing: 1.17.0
  - files_trashbin: 1.15.0
  - files_versions: 1.18.0
  - files_versions_s3: 0.1.9
  - guests: 2.3.0
  - integration_google: 1.0.9
  - integration_mastodon: 1.0.3
  - lookup_server_connector: 1.13.0
  - memories: 4.9.3
  - metadata: 0.17.0
  - music: 1.7.0
  - notes: 4.6.0
  - oauth2: 1.13.0
  - provisioning_api: 1.15.0
  - ransomware_protection: 1.14.0
  - secrets: 1.0.1
  - settings: 1.7.0
  - sharebymail: 1.15.0
  - sharerenamer: 3.1.0
  - sms_relentless: 1.1.4
  - systemtags: 1.15.0
  - tasks: 0.14.5
  - theming: 2.0.0
  - twofactor_backupcodes: 1.14.0
  - updatenotification: 1.15.0
  - user_status: 1.5.0
  - weather_status: 1.5.0
  - workflowengine: 2.7.0
Disabled:
  - breezedark: 24.0.2
  - calendar: 4.2.0-rc.1
  - deck: 1.8.2
  - encryption
  - extract: 1.3.5
  - files_reader: 1.5.3
  - files_readmemd: 1.2.2
  - geoblocker: 0.5.7
  - richdocuments: 7.0.2
  - richdocumentscode: 22.5.802
  - spreed: 15.0.2
  - testing
  - unsplash: 2.0.1
  - user_ldap

Nextcloud Signing status

Integrity checker has been disabled. Integrity cannot be verified.

Nextcloud Logs

Logging disabled

Additional info

Ran into this issue on both v24 and v25. Previously had a v24 Snap install. The workaround was to just let Nextcloud create a new bucket. I didn't realize Nextcloud put the bucket in eu-west-1. After moving to a v25 manual install, I tried to fix this by creating a us-east-1 bucket in AWS and switch to that. Ran into the same issue; however, this time I did try to explicitly set the region to us-east-1, which fixed the error.

Source code: This is where the eu-west-1 default is coming from. Based on that, it doesn't search for the bucket, to just defaults to eu-west-1 if the bucket region is not supplied: https://github.com/nextcloud/server/blob/e4e20bf40ad8ca139655b36a6efa2b1710ae50f0/lib/private/Files/ObjectStore/S3ConnectionTrait.php#L86

Seems like GetBucketLocation, or HeadBucket would be needed if the app was to figure out the region of pre-existing buckets before creating S3Client. Those APIs use the "List buckets" permission to return the bucket region (user must own the bucket). I'm not 100% sure how it would fit into S3ConnectionTrait.php because connections to AWS are made using S3Client and the S3Client needs the region for the constructor to work.

@icewind1991 it looks like you've worked on the External Storage app before. You might be able to speak to whether updating the s3 code to check for bucket location is worth the dev time, or updating documentation about this gotcha would be better.

nextcloud / server