[Bug]: S3 ObjectStorage putObject rate limit

⚠️ This issue respects the following points: ⚠️

[X] This is a bug, not a question or a configuration/webserver/proxy issue.
[X] This issue is not already reported on Github OR Nextcloud Community Forum (I've searched it).
[X] Nextcloud Server is up to date. See Maintenance and Release Schedule for supported versions.
[X] I agree to follow Nextcloud's Code of Conduct.

Bug description

I installed a fresh nextcloud deployment with a S3 primary storage over a own storj-gateway (minio-fork).

Single file operation small and large working fine, but there are rate limit issues with some components like richdoccuments.

On template file creation, the file is first created (file touch operation) and after the content is written, see: (https://github.com/nextcloud/richdocuments/blob/0e585e70a0f398bd79edc07a00d05b7b0868bd2c/lib/TemplateManager.php#L220-L221)

This produces two putObject operation on the S3 storj backend in less then one second Storj has a limit with 1 write per second to the same object name. see: (https://docs.storj.io/learn/concepts/limits#credit-card-payment-method)

The creation of the first template file fails with 429 Too Many Requests that is not handled/retried property.

Steps to reproduce

Use storj as the S3 primary storage
First login or exec php8.3 occ richdocuments:update-empty-templates

Expected behavior

The newfile() + writecontent() tween should not produce two putObject requests on the S3 and/or S3 429 Too Many Requests should be handled better and/or set throttle rates in the config

Installation method

Community Manual installation with Archive

Nextcloud Server version

Operating system

Debian/Ubuntu

PHP engine version

PHP 8.3

Web server

Apache (supported)

Database engine version

MariaDB

Is this bug present after an update or on a fresh install?

Fresh Nextcloud Server install

Are you using the Nextcloud Server Encryption module?

Encryption is Disabled

What user-backends are you using?

[X] Default user-backend (database)
[ ] LDAP/ Active Directory
[ ] SSO - SAML
[X] Other

Configuration report

{
    "system": {
        "instanceid": "***REMOVED SENSITIVE VALUE***",
        "passwordsalt": "***REMOVED SENSITIVE VALUE***",
        "secret": "***REMOVED SENSITIVE VALUE***",
        "trusted_domains": [
            "mydomain.test"
        ],
        "datadirectory": "***REMOVED SENSITIVE VALUE***",
        "cache_path": "\/home\/webuser\/tmp\/cache",
        "dbtype": "mysql",
        "version": "29.0.2.2",
        "overwrite.cli.url": "https:\/\/mydomain.test",
        "htaccess.RewriteBase": "\/",
        "dbname": "***REMOVED SENSITIVE VALUE***",
        "dbhost": "***REMOVED SENSITIVE VALUE***",
        "dbport": "",
        "dbtableprefix": "oc_",
        "mysql.utf8mb4": true,
        "dbuser": "***REMOVED SENSITIVE VALUE***",
        "dbpassword": "***REMOVED SENSITIVE VALUE***",
        "installed": true,
        "objectstore": {
            "class": "\\OC\\Files\\ObjectStore\\S3",
            "arguments": {
                "bucket": "nextcloud",
                "hostname": "127.0.0.1",
                "key": "***REMOVED SENSITIVE VALUE***",
                "secret": "***REMOVED SENSITIVE VALUE***",
                "port": 7777,
                "use_ssl": false,
                "use_path_style": true,
                "uploadPartSize": 67108864,
                "putSizeLimit": 67108864,
                "concurrency": 1
            }
        },
        "memcache.locking": "\\OC\\Memcache\\Redis",
        "memcache.distributed": "\\OC\\Memcache\\Redis",
        "memcache.local": "\\OC\\Memcache\\APCu",
        "redis": {
            "host": "***REMOVED SENSITIVE VALUE***",
            "port": 0
        },
        "mail_smtpmode": "smtp",
        "mail_sendmailmode": "smtp",
        "mail_from_address": "***REMOVED SENSITIVE VALUE***",
        "mail_domain": "***REMOVED SENSITIVE VALUE***",
        "mail_smtphost": "***REMOVED SENSITIVE VALUE***",
        "mail_smtpport": "587",
        "mail_smtpauth": 1,
        "mail_smtpname": "***REMOVED SENSITIVE VALUE***",
        "mail_smtppassword": "***REMOVED SENSITIVE VALUE***",
        "maintenance": false,
        "data-fingerprint": "e94b1423e687644a749a63f853521ad9"
    }
}

List of activated Apps

Enabled:
  - activity: 2.21.1
  - calendar: 4.7.6
  - circles: 29.0.0-dev
  - cloud_federation_api: 1.12.0
  - comments: 1.19.0
  - contacts: 6.0.0
  - contactsinteraction: 1.10.0
  - dashboard: 7.9.0
  - dav: 1.30.1
  - external: 5.4.0
  - federatedfilesharing: 1.19.0
  - federation: 1.19.0
  - files: 2.1.0
  - files_downloadlimit: 2.0.0
  - files_pdfviewer: 2.10.0
  - files_reminders: 1.2.0
  - files_sharing: 1.21.0
  - files_trashbin: 1.19.0
  - files_versions: 1.22.0
  - firstrunwizard: 2.18.0
  - logreader: 2.14.0
  - lookup_server_connector: 1.17.0
  - mail: 3.7.1
  - nextcloud_announcements: 1.18.0
  - notes: 4.10.0
  - notifications: 2.17.0
  - oauth2: 1.17.0
  - password_policy: 1.19.0
  - photos: 2.5.0
  - privacy: 1.13.0
  - provisioning_api: 1.19.0
  - recommendations: 2.1.0
  - related_resources: 1.4.0
  - richdocuments: 8.4.3
  - serverinfo: 1.19.0
  - settings: 1.12.0
  - sharebymail: 1.19.0
  - spreed: 19.0.3
  - support: 1.12.0
  - survey_client: 1.17.0
  - systemtags: 1.19.0
  - text: 3.10.0
  - theming: 2.4.0
  - twofactor_backupcodes: 1.18.0
  - updatenotification: 1.19.1
  - user_oidc: 5.0.2
  - user_status: 1.9.0
  - viewer: 2.3.0
  - weather_status: 1.9.0
  - workflowengine: 2.11.0
Disabled:
  - admin_audit: 1.19.0
  - bruteforcesettings: 2.9.0
  - encryption: 2.17.0
  - files_external: 1.21.0
  - suspicious_login: 7.0.0
  - twofactor_totp: 11.0.0-dev
  - user_ldap: 1.20.0

Nextcloud Signing status

No response

Nextcloud Logs

No response

Additional info

I worked around the issue now by adding a 1sec sleep after file touch operation in /lib/private/Files/View.php

nextcloud / server