nextcloud / server

☁️ Nextcloud server, a safe home for all your data
https://nextcloud.com
GNU Affero General Public License v3.0
26.92k stars 4.01k forks source link

[Bug]: files_external - Undefined array key \"key\" w/ IAM instance auth to S3 #46400

Open markuman opened 2 months ago

markuman commented 2 months ago

⚠️ This issue respects the following points: ⚠️

Bug description

Using AWS S3 as external storage using EC2 Instance Role. So "Authentication" is set to None.
This results in millions of Undefined array key "key" errors in data/nextcloud.log

[PHP] Error: Undefined array key "key" at /var/www/html/apps/files_external/lib/Lib/Storage/AmazonS3.php#83
    GET /ocs/v1.php/cloud/users/dahlmant
    from 10.112.60.178 by admin at 10.07.2024, 13:57:00

Furthermore occ files:scan <user> results also in errors

Starting scan for user 1 out of 1 (exampleuser)
Error during scan: Undefined array key "key"
+---------+-------+-----+---------+---------+--------+--------------+
| Folders | Files | New | Updated | Removed | Errors | Elapsed time |
+---------+-------+-----+---------+---------+--------+--------------+
| 16      | 47    | 0   | 3       | 0       | 1      | 00:00:00     |
+---------+-------+-----+---------+---------+--------+--------------+

Read/Write operations on the external storage works without any problems.

Steps to reproduce

  1. Host Nextcloud on AWS EC2 with IAM Instance role to get R/W permissions to a S3 bucket
  2. Configure external S3 storage with authentication none
  3. log file will be flooted

Expected behavior

Correct handling of authentication type none. No errors in the log nor with occ files:scan

Installation method

Community Docker image

Nextcloud Server version

29

Operating system

Debian/Ubuntu

PHP engine version

PHP 8.3

Web server

Apache (supported)

Database engine version

MariaDB

Is this bug present after an update or on a fresh install?

Fresh Nextcloud Server install

Are you using the Nextcloud Server Encryption module?

Encryption is Disabled

What user-backends are you using?

Configuration report

{
    "system": {
        "htaccess.RewriteBase": "\/",
        "memcache.local": "\\OC\\Memcache\\APCu",
        "apps_paths": [
            {
                "path": "\/var\/www\/html\/apps",
                "url": "\/apps",
                "writable": false
            },
            {
                "path": "\/var\/www\/html\/custom_apps",
                "url": "\/custom_apps",
                "writable": true
            }
        ],
        "memcache.distributed": "\\OC\\Memcache\\Redis",
        "memcache.locking": "\\OC\\Memcache\\Redis",
        "redis": {
            "host": "***REMOVED SENSITIVE VALUE***",
            "password": "***REMOVED SENSITIVE VALUE***",
            "port": 6379
        },
        "upgrade.disable-web": true,
        "passwordsalt": "***REMOVED SENSITIVE VALUE***",
        "secret": "***REMOVED SENSITIVE VALUE***",
        "trusted_domains": [
            "localhost",
            "cloud.aws.XXXX.de",
            "cloud.XXXX.de"
        ],
        "datadirectory": "***REMOVED SENSITIVE VALUE***",
        "dbtype": "mysql",
        "version": "29.0.3.4",
        "overwrite.cli.url": "https:\/\/cloud.XXXX.de",
        "dbname": "***REMOVED SENSITIVE VALUE***",
        "dbhost": "***REMOVED SENSITIVE VALUE***",
        "dbport": "",
        "dbtableprefix": "oc_",
        "mysql.utf8mb4": true,
        "dbuser": "***REMOVED SENSITIVE VALUE***",
        "dbpassword": "***REMOVED SENSITIVE VALUE***",
        "installed": true,
        "instanceid": "***REMOVED SENSITIVE VALUE***",
        "oidc_login_logout_url": "https:\/\/cloud.XXXX.de\/apps\/oidc_login\/oidc",
        "oidc_login_end_session_redirect": true,
        "oidc_login_auto_redirect": false,
        "oidc_login_redir_fallback": true,
        "oidc_login_attributes": {
            "id": "preferred_username",
            "mail": "email"
        },
        "overwriteprotocol": "https",
        "oidc_login_button_text": "Login with keycloak",
        "oidc_login_disable_registration": false,
        "dbdriveroptions": {
            "PDO::MYSQL_ATTR_SSL_CA": "\/dev\/null",
            "PDO::MYSQL_ATTR_SSL_VERIFY_SERVER_CERT": false
        },
        "default_phone_region": "DE",
        "maintenance_window_start": 1,
        "mail_from_address": "***REMOVED SENSITIVE VALUE***",
        "mail_smtpmode": "smtp",
        "mail_sendmailmode": "smtp",
        "mail_domain": "***REMOVED SENSITIVE VALUE***",
        "mail_smtphost": "***REMOVED SENSITIVE VALUE***",
        "mail_smtpport": 587,
        "mail_smtpauth": 1,
        "mail_smtpname": "***REMOVED SENSITIVE VALUE***",
        "mail_smtppassword": "***REMOVED SENSITIVE VALUE***",
        "forwarded_for_headers": [
            "HTTP_X_FORWARDED_FOR"
        ],
        "oidc_login_client_id": "nextcloud",
        "oidc_login_client_secret": "1234asdf",
        "oidc_login_provider_url": "https:\/\/keycloak.XXXX.de\/realms\/intern",
        "hide_login_form": true,
        "allow_local_remote_servers": true,
        "loglevel": 2,
        "maintenance": false,
        "default_language": "de",
        "default_locale": "de_DE"
    }
}

List of activated Apps

Enabled:
  - activity: 2.21.1
  - announcementcenter: 6.8.1
  - appointments: 2.1.6
  - bruteforcesettings: 2.9.0
  - calendar: 4.7.10
  - cfg_share_links: 5.1.0
  - circles: 29.0.0-dev
  - cloud_federation_api: 1.12.0
  - collectives: 2.12.0
  - comments: 1.19.0
  - contactsinteraction: 1.10.0
  - dav: 1.30.1
  - deck: 1.13.1
  - drawio: 3.0.2
  - external: 5.4.0
  - federatedfilesharing: 1.19.0
  - federation: 1.19.0
  - files: 2.1.0
  - files_downloadlimit: 2.0.0
  - files_external: 1.21.0
  - files_pdfviewer: 2.10.0
  - files_reminders: 1.2.0
  - files_sharing: 1.21.0
  - files_trashbin: 1.19.0
  - files_versions: 1.22.0
  - firstrunwizard: 2.18.0
  - groupfolders: 17.0.1
  - logreader: 2.14.0
  - lookup_server_connector: 1.17.0
  - nextcloud_announcements: 1.18.0
  - notifications: 2.17.0
  - oauth2: 1.17.0
  - password_policy: 1.19.0
  - privacy: 1.13.0
  - provisioning_api: 1.19.0
  - recommendations: 2.1.0
  - related_resources: 1.4.0
  - serverinfo: 1.19.0
  - settings: 1.12.0
  - sharebymail: 1.19.0
  - support: 1.12.0
  - survey_client: 1.17.0
  - systemtags: 1.19.0
  - text: 3.10.1
  - theming: 2.4.0
  - twofactor_backupcodes: 1.18.0
  - updatenotification: 1.19.1
  - user_oidc: 5.0.3
  - user_status: 1.9.0
  - viewer: 2.3.0
  - weather_status: 1.9.0
  - workflowengine: 2.11.0
Disabled:
  - admin_audit: 1.19.0
  - dashboard: 7.9.0 (installed 7.8.0)
  - encryption: 2.17.0
  - photos: 2.5.0 (installed 2.4.0)
  - suspicious_login: 7.0.0
  - twofactor_totp: 11.0.0-dev
  - user_ldap: 1.20.0

Nextcloud Signing status

No errors have been found.

Nextcloud Logs

{"reqId":"Rv8UjniG7cMZ20TFo63Q","level":3,"time":"2024-07-09T08:37:36+00:00","remoteAddr":"10.112.60.178","user":"admin","app":"PHP","method":"GET","url":"/ocs/v1.php/cloud/users/hochschr","message":"Undefined array key \"key\" at /var/www/html/apps/files_external/lib/Lib/Storage/AmazonS3.php#83","userAgent":"--","version":"29.0.3.4","data":{"app":"PHP"}}

Additional info

No response

solracsf commented 2 months ago

Can you change that file /var/www/html/apps/files_external/lib/Lib/Storage/AmazonS3.php line 83 so it reads:

$this->id = 'amazon::external::' . md5($this->params['hostname'] . ':' . $this->params['bucket'] . ':' . ($this->params['key'] ?? ''));

and retry?

markuman commented 2 months ago

@solracsf works for me.