nextcloud / server

☁️ Nextcloud server, a safe home for all your data
https://nextcloud.com
GNU Affero General Public License v3.0
26.63k stars 3.99k forks source link

[Bug]: Drag and Drop folder failed in a load-balanced System #46170

Open dsisysteme opened 2 months ago

dsisysteme commented 2 months ago

⚠️ This issue respects the following points: ⚠️

Bug description

We have noticed that in a load-balanced environment without Sticky Sessions, dragging and dropping a folder containing multiple files no longer works. In the case of an environment with 2 application servers, only half of the files are uploaded correctly. The web interface displays error messages saying "Operation blocked by access control."

We can see that one of the nodes is returning 403 errors on file uploads while the second one handles them correctly. Based on repeated tests, it's clear that this issue is not specific to any particular node. When we isolate either node 1 or node 2, drag and drop functionality works.

For your information, this works perfectly in 27.1.4 under the same conditions.

Steps to reproduce

  1. load-balanced environment: 1 reverse proxy, 2 application servers, 1 MariaDB server + Redis
  2. configure the reverse proxy to load-balance requests in a round-robin fashion across the 2 application servers without sticky sessions
  3. configure PHP to store sessions in Redis
  4. configure Nextcloud with local memcache on APCu and distributed memcache on Redis
  5. Drag and Drop a folder (testDrag) which contain 10 files (1 to 10)

Expected behavior

The folder and all files should be uploaded without any error messages.

Installation method

Community Manual installation with Archive

Nextcloud Server version

29

Operating system

Debian/Ubuntu

PHP engine version

PHP 8.3

Web server

Apache (supported)

Database engine version

MariaDB

Is this bug present after an update or on a fresh install?

Fresh Nextcloud Server install

Are you using the Nextcloud Server Encryption module?

Encryption is Disabled

What user-backends are you using?

Configuration report

{
    "system": {
        "instanceid": "***REMOVED SENSITIVE VALUE***",
        "passwordsalt": "***REMOVED SENSITIVE VALUE***",
        "secret": "***REMOVED SENSITIVE VALUE***",
        "trusted_proxies": "***REMOVED SENSITIVE VALUE***",
        "trusted_domains": [
            "***REMOVED SENSITIVE VALUE***"
        ],
        "datadirectory": "***REMOVED SENSITIVE VALUE***",
        "overwrite.cli.url": "***REMOVED SENSITIVE VALUE***",
        "overwriteprotocol": "https",
        "dbtype": "mysql",
        "version": "29.0.3.4",
        "dbname": "***REMOVED SENSITIVE VALUE***",
        "dbhost": "***REMOVED SENSITIVE VALUE***",
        "dbport": "",
        "dbtableprefix": "oc_",
        "mysql.utf8mb4": true,
        "dbuser": "***REMOVED SENSITIVE VALUE***",
        "dbpassword": "***REMOVED SENSITIVE VALUE***",
        "installed": true,
        "ldapIgnoreNamingRules": false,
        "ldapProviderFactory": "\\OCA\\User_LDAP\\LDAPProviderFactory",
        "ldapUserCleanupInterval": 0,
        "memcache.local": "\\OC\\Memcache\\APCu",
        "memcache.distributed": "\\OC\\Memcache\\Redis",
        "memcache.locking": "\\OC\\Memcache\\Redis",
        "filelocking.enabled": true,
        "redis": {
            "host": "***REMOVED SENSITIVE VALUE***",
            "port": 6379,
            "dbindex": 1
        },
        "filesystem_check_changes": 0,
        "enable_avatars": false,
        "mail_from_address": "***REMOVED SENSITIVE VALUE***",
        "mail_smtpmode": "sendmail",
        "mail_smtpauthtype": "LOGIN",
        "mail_domain": "***REMOVED SENSITIVE VALUE***",
        "share_folder": "\/Shared",
        "cron_log": true,
        "loglevel": "2",
        "maintenance": false,
        "trashbin_retention_obligation": 30,
        "trashbin_auto_expire": true,
        "logfile": "\/var\/log\/nextcloud\/nextcloud.log",
        "knowledgebaseenabled": false,
        "sharing.maxAutocompleteResults": 20,
        "sharing.minSearchStringLength": 3,
        "default_phone_region": "FR",
        "maintenance_window_start": 3,
        "activity_expire_days": "180",
        "simpleSignUpLink.shown": false,
        "default_timezone": "Europe\/Paris",
        "logtimezone": "Europe\/Paris",
        "enable_previews": true,
        "enabledPreviewProviders": [
            "OC\\Preview\\BMP",
            "OC\\Preview\\GIF",
            "OC\\Preview\\JPEG",
            "OC\\Preview\\MP3",
            "OC\\Preview\\PNG",
            "OC\\Preview\\XBitmap"
        ]
    }
}

List of activated Apps

Enabled:
  - activity: 2.21.1
  - cloud_federation_api: 1.12.0
  - comments: 1.19.0
  - contactsinteraction: 1.10.0
  - dav: 1.30.1
  - federatedfilesharing: 1.19.0
  - files: 2.1.0
  - files_downloadlimit: 2.0.0
  - files_pdfviewer: 2.10.0
  - files_sharing: 1.21.0
  - files_trashbin: 1.19.0
  - files_versions: 1.22.0
  - lookup_server_connector: 1.17.0
  - notifications: 2.17.0
  - oauth2: 1.17.0
  - photos: 2.5.0
  - provisioning_api: 1.19.0
  - richdocuments: 8.4.3
  - serverinfo: 1.19.0
  - settings: 1.12.0
  - sharebymail: 1.19.0
  - text: 3.10.1
  - theming: 2.4.0
  - twofactor_backupcodes: 1.18.0
  - user_ldap: 1.20.0
  - user_saml: 6.1.3
  - viewer: 2.3.0
  - workflowengine: 2.11.0
Disabled:
  - admin_audit: 1.19.0
  - bruteforcesettings: 2.9.0
  - circles: 29.0.0-dev (installed 25.0.0)
  - dashboard: 7.9.0 (installed 7.0.0)
  - encryption: 2.17.0
  - federation: 1.19.0 (installed 1.3.0)
  - files_external: 1.21.0
  - files_reminders: 1.2.0 (installed 1.0.0)
  - firstrunwizard: 2.18.0 (installed 2.14.0)
  - logreader: 2.14.0 (installed 2.10.0)
  - nextcloud_announcements: 1.18.0 (installed 1.14.0)
  - password_policy: 1.19.0 (installed 1.15.0)
  - privacy: 1.13.0 (installed 1.9.0)
  - recommendations: 2.1.0 (installed 1.4.0)
  - related_resources: 1.4.0 (installed 1.0.4)
  - support: 1.12.0 (installed 1.8.0)
  - survey_client: 1.17.0 (installed 1.13.0)
  - suspicious_login: 7.0.0
  - systemtags: 1.19.0 (installed 1.3.0)
  - twofactor_totp: 11.0.0-dev
  - updatenotification: 1.19.1 (installed 1.17.0)
  - user_status: 1.9.0 (installed 1.0.1)
  - weather_status: 1.9.0 (installed 1.7.0)

Nextcloud Signing status

No response

Nextcloud Logs

No response

Additional info

Browser log (Firefox 127.0):

[DEBUG] files: Initial templates folder 
Object { app: "files", uid: "***LOGIN***", level: "2", templatesPath: false }
ConsoleLogger.js:65:18
[DEBUG] files: FileActions initialized 
Object { app: "files", uid: "***LOGIN***", level: "2" }
index.mjs:45:16
[DEBUG] files: NewFileMenu initialized 
Object { app: "files", uid: "***LOGIN***", level: "2" }
index.mjs:45:16
[DEBUG] files: Generating favorites view 
Object { app: "files", uid: "***LOGIN***", level: "2", favoriteFolders: [] }
ConsoleLogger.js:65:18
[DEBUG] files: Navigation service initialized 
Object { app: "files", uid: "***LOGIN***", level: "2" }
index.mjs:45:16
[INFO] activity: Activity API registered 
Object { app: "activity", uid: "***LOGIN***", level: "2" }
logger-BVTvLOjn.mjs:2:30195
Unified search initialized! UnifiedSearch.vue:54
OCA.Files.Settings initialized Settings.js:29:10
[DEBUG] files: Navigation mounted. Showing requested view 
Object { app: "files", uid: "***LOGIN***", level: "2", view: {…} }
ConsoleLogger.js:65:18
[DEBUG] uploader: Destination set 
Object { app: "uploader", uid: "***LOGIN***", level: "2", folder: {…} }
ConsoleLogger.js:65:18
[DEBUG] uploader: Upload workspace initialized 
Object { app: "uploader", uid: "***LOGIN***", level: "2", destination: {…}, root: "https://***URL-PLATEFORME***/remote.php/dav/files/***LOGIN***", isPublic: false, maxChunksSize: 524288000 }
ConsoleLogger.js:65:18
OCA.Files.Sidebar initialized Sidebar.js:36:10
OCA.Viewer initialized viewer-main.mjs:2187:581
OCA.Sharing.ShareSearch initialized ShareSearch.js:33:10
OCA.Sharing.ExternalLinkActions initialized ExternalLinkActions.js:33:10
OCA.Sharing.ExternalShareActions initialized ExternalShareActions.js:33:10
OCA.Comments.View initialized comments-app.js:32:8
[INFO] files: Initializing unified search plugin: folder search from files app 
Object { app: "files", uid: "***LOGIN***", level: "2" }
ConsoleLogger.js:68:18
[INFO] viewer: 6 viewer handlers registered 
Object { app: "viewer", uid: "***LOGIN***", level: "2", handlers: (6) […] }
viewer-main.mjs:44:184629
session heartbeat polling started session-heartbeat.js:103:9
[DEBUG] activity: Registered new sidebar action 
Object { app: "activity", uid: "***LOGIN***", level: "2" }
logger-BVTvLOjn.mjs:2:30110
[DEBUG] activity: Registered new sidebar actions factory 
Object { app: "activity", uid: "***LOGIN***", level: "2" }
logger-BVTvLOjn.mjs:2:30110
[INFO] comments: Comments plugin registered for Activity sidebar action 
Object { app: "comments", uid: "***LOGIN***", level: "2" }
ConsoleLogger.js:68:18
Registering notifications container as a menu NotificationsApp.vue:237
Notifications permissions denied NotificationsApp.vue:486
Polling interval updated to 30000 NotificationsApp.vue:443
Started background fetcher as session_keepalive is enabled NotificationsApp.vue:293
[DEBUG] core: this browser is officially supported ! 🚀 
Object { app: "core", uid: "***LOGIN***", level: "2" }
ConsoleLogger.js:65:18
Search providers 
Array(5) [ {…}, {…}, {…}, {…}, {…} ]
UnifiedSearchModal.vue:275
Contacts 
Array [ {…} ]
UnifiedSearchModal.vue:279
Got notification data, restoring default polling interval. NotificationsApp.vue:399
[DEBUG] files: SW registered:  
Object { app: "files", uid: "***LOGIN***", level: "2", registration: ServiceWorkerRegistration }
ConsoleLogger.js:65:18
[DEBUG] files: Fetched contents 
Object { app: "files", uid: "***LOGIN***", level: "2", dir: "/", folder: {…}, contents: (8) […] }
ConsoleLogger.js:65:18
[DEBUG] files: Directory contents changed 
Object { app: "files", uid: "***LOGIN***", level: "2", view: {…}, folder: {…}, contents: (8) […] }
ConsoleLogger.js:65:18
[DEBUG] files: FileListHeaders initialized 
Object { app: "files", uid: "***LOGIN***", level: "2" }
index.mjs:45:16
[DEBUG] uploader: Destination set 
Object { app: "uploader", uid: "***LOGIN***", level: "2", folder: {…} }
ConsoleLogger.js:65:18
[DEBUG] uploader: UploadPicker initialised 
Object { app: "uploader", uid: "***LOGIN***", level: "2" }
ConsoleLogger.js:65:18
[DEBUG] files: VirtualList: resizeObserver updated 
Object { app: "files", uid: "***LOGIN***", level: "2" }
ConsoleLogger.js:65:18
[DEBUG] files: Cancelled previous ongoing fetch 
Object { app: "files", uid: "***LOGIN***", level: "2" }
ConsoleLogger.js:65:18
[DEBUG] files: Fetched contents 
Object { app: "files", uid: "***LOGIN***", level: "2", dir: "/", folder: {…}, contents: (8) […] }
ConsoleLogger.js:65:18
[DEBUG] files: Directory contents changed 
Object { app: "files", uid: "***LOGIN***", level: "2", view: {…}, folder: {…}, contents: (8) […] }
ConsoleLogger.js:65:18
[DEBUG] uploader: Destination set 
Object { app: "uploader", uid: "***LOGIN***", level: "2", folder: {…} }
ConsoleLogger.js:65:18
[DEBUG] files: Updating selection 
Object { app: "files", uid: "***LOGIN***", level: "2", selection: (1) […] }
ConsoleLogger.js:65:18
[DEBUG] files: Directory contents changed 
Object { app: "files", uid: "***LOGIN***", level: "2", view: {…}, folder: {…}, contents: (7) […] }
ConsoleLogger.js:65:18
[DEBUG] files: VirtualList: resizeObserver updated 
Object { app: "files", uid: "***LOGIN***", level: "2" }
ConsoleLogger.js:65:18
[DEBUG] files: Handling recursive file tree 
Object { app: "files", uid: "***LOGIN***", level: "2", entry: "testDrag" }
ConsoleLogger.js:65:18
[DEBUG] files: Dropped 
Object { app: "files", uid: "***LOGIN***", level: "2", event: drop, folder: {…}, fileTree: File }
ConsoleLogger.js:65:18
[DEBUG] files: Uploading files to / 
Object { app: "files", uid: "***LOGIN***", level: "2", root: File, contents: (1) […] }
ConsoleLogger.js:65:18
Processing directory 
Object { relativePath: "/testDrag" }
DropService.ts:121:28
XHRPROPFIND
https://***URL-PLATEFORME***/remote.php/dav/files/***LOGIN***/testDrag
[HTTP/1.1 404 Not Found 30ms]

[DEBUG] files: Directory does not exist, creating it 
Object { app: "files", uid: "***LOGIN***", level: "2", absolutePath: "/files/***LOGIN***/testDrag" }
ConsoleLogger.js:65:18
XHRPROPFIND
https://***URL-PLATEFORME***/remote.php/dav/files/***LOGIN***/testDrag
[HTTP/1.1 404 Not Found 28ms]

[DEBUG] files: Directory contents changed 
Object { app: "files", uid: "***LOGIN***", level: "2", view: {…}, folder: {…}, contents: (8) […] }
ConsoleLogger.js:65:18
[DEBUG] files: Uploading file to /testDrag/4 
Object { app: "files", uid: "***LOGIN***", level: "2", file: File }
ConsoleLogger.js:65:18
[DEBUG] uploader: Uploading 4 to https://***URL-PLATEFORME***/remote.php/dav/files/***LOGIN***/testDrag/4 
Object { app: "uploader", uid: "***LOGIN***", level: "2" }
ConsoleLogger.js:65:18
[DEBUG] uploader: Initializing regular upload 
Object { app: "uploader", uid: "***LOGIN***", level: "2", file: File, upload: {…} }
ConsoleLogger.js:65:18
[DEBUG] files: Uploading file to /testDrag/10 
Object { app: "files", uid: "***LOGIN***", level: "2", file: File }
ConsoleLogger.js:65:18
[DEBUG] uploader: Uploading 10 to https://***URL-PLATEFORME***/remote.php/dav/files/***LOGIN***/testDrag/10 
Object { app: "uploader", uid: "***LOGIN***", level: "2" }
ConsoleLogger.js:65:18
[DEBUG] uploader: Initializing regular upload 
Object { app: "uploader", uid: "***LOGIN***", level: "2", file: File, upload: {…} }
ConsoleLogger.js:65:18
[DEBUG] files: Uploading file to /testDrag/2 
Object { app: "files", uid: "***LOGIN***", level: "2", file: File }
ConsoleLogger.js:65:18
[DEBUG] uploader: Uploading 2 to https://***URL-PLATEFORME***/remote.php/dav/files/***LOGIN***/testDrag/2 
Object { app: "uploader", uid: "***LOGIN***", level: "2" }
ConsoleLogger.js:65:18
[DEBUG] uploader: Initializing regular upload 
Object { app: "uploader", uid: "***LOGIN***", level: "2", file: File, upload: {…} }
ConsoleLogger.js:65:18
[DEBUG] files: Uploading file to /testDrag/3 
Object { app: "files", uid: "***LOGIN***", level: "2", file: File }
ConsoleLogger.js:65:18
[DEBUG] uploader: Uploading 3 to https://***URL-PLATEFORME***/remote.php/dav/files/***LOGIN***/testDrag/3 
Object { app: "uploader", uid: "***LOGIN***", level: "2" }
ConsoleLogger.js:65:18
[DEBUG] uploader: Initializing regular upload 
Object { app: "uploader", uid: "***LOGIN***", level: "2", file: File, upload: {…} }
ConsoleLogger.js:65:18
[DEBUG] files: Uploading file to /testDrag/1 
Object { app: "files", uid: "***LOGIN***", level: "2", file: File }
ConsoleLogger.js:65:18
[DEBUG] uploader: Uploading 1 to https://***URL-PLATEFORME***/remote.php/dav/files/***LOGIN***/testDrag/1 
Object { app: "uploader", uid: "***LOGIN***", level: "2" }
ConsoleLogger.js:65:18
[DEBUG] uploader: Initializing regular upload 
Object { app: "uploader", uid: "***LOGIN***", level: "2", file: File, upload: {…} }
ConsoleLogger.js:65:18
[DEBUG] files: Uploading file to /testDrag/9 
Object { app: "files", uid: "***LOGIN***", level: "2", file: File }
ConsoleLogger.js:65:18
[DEBUG] uploader: Uploading 9 to https://***URL-PLATEFORME***/remote.php/dav/files/***LOGIN***/testDrag/9 
Object { app: "uploader", uid: "***LOGIN***", level: "2" }
ConsoleLogger.js:65:18
[DEBUG] uploader: Initializing regular upload 
Object { app: "uploader", uid: "***LOGIN***", level: "2", file: File, upload: {…} }
ConsoleLogger.js:65:18
[DEBUG] files: Uploading file to /testDrag/6 
Object { app: "files", uid: "***LOGIN***", level: "2", file: File }
ConsoleLogger.js:65:18
[DEBUG] uploader: Uploading 6 to https://***URL-PLATEFORME***/remote.php/dav/files/***LOGIN***/testDrag/6 
Object { app: "uploader", uid: "***LOGIN***", level: "2" }
ConsoleLogger.js:65:18
[DEBUG] uploader: Initializing regular upload 
Object { app: "uploader", uid: "***LOGIN***", level: "2", file: File, upload: {…} }
ConsoleLogger.js:65:18
[DEBUG] files: Uploading file to /testDrag/7 
Object { app: "files", uid: "***LOGIN***", level: "2", file: File }
ConsoleLogger.js:65:18
[DEBUG] uploader: Uploading 7 to https://***URL-PLATEFORME***/remote.php/dav/files/***LOGIN***/testDrag/7 
Object { app: "uploader", uid: "***LOGIN***", level: "2" }
ConsoleLogger.js:65:18
[DEBUG] uploader: Initializing regular upload 
Object { app: "uploader", uid: "***LOGIN***", level: "2", file: File, upload: {…} }
ConsoleLogger.js:65:18
[DEBUG] files: Uploading file to /testDrag/5 
Object { app: "files", uid: "***LOGIN***", level: "2", file: File }
ConsoleLogger.js:65:18
[DEBUG] uploader: Uploading 5 to https://***URL-PLATEFORME***/remote.php/dav/files/***LOGIN***/testDrag/5 
Object { app: "uploader", uid: "***LOGIN***", level: "2" }
ConsoleLogger.js:65:18
[DEBUG] uploader: Initializing regular upload 
Object { app: "uploader", uid: "***LOGIN***", level: "2", file: File, upload: {…} }
ConsoleLogger.js:65:18
[DEBUG] files: Uploading file to /testDrag/8 
Object { app: "files", uid: "***LOGIN***", level: "2", file: File }
ConsoleLogger.js:65:18
[DEBUG] uploader: Uploading 8 to https://***URL-PLATEFORME***/remote.php/dav/files/***LOGIN***/testDrag/8 
Object { app: "uploader", uid: "***LOGIN***", level: "2" }
ConsoleLogger.js:65:18
[DEBUG] uploader: Initializing regular upload 
Object { app: "uploader", uid: "***LOGIN***", level: "2", file: File, upload: {…} }
ConsoleLogger.js:65:18
XHRPUT
https://***URL-PLATEFORME***/remote.php/dav/files/***LOGIN***/testDrag/4
[HTTP/1.1 403 Forbidden 28ms]

XHRPUT
https://***URL-PLATEFORME***/remote.php/dav/files/***LOGIN***/testDrag/2
[HTTP/1.1 403 Forbidden 44ms]

[ERROR] uploader: Failed uploading 4 
Object { app: "uploader", uid: "***LOGIN***", level: "2", error: {…}, file: File, upload: {…} }
ConsoleLogger.js:74:18
XHRPUT
https://***URL-PLATEFORME***/remote.php/dav/files/***LOGIN***/testDrag/3
[HTTP/1.1 403 Forbidden 26ms]

[ERROR] uploader: Failed uploading 2 
Object { app: "uploader", uid: "***LOGIN***", level: "2", error: {…}, file: File, upload: {…} }
ConsoleLogger.js:74:18
    value ConsoleLogger.js:74
    value ConsoleLogger.js:100
    i index-DM2X1kc6.mjs:275
    add index.js:180
    #x index.js:118
    #L index.js:149
    start index.js:218
    start index-DM2X1kc6.mjs:223
    Us DropService.ts:143
    onDrop DragAndDropNotice.vue:120
    VueJS 27
[ERROR] uploader: Failed uploading 3 
Object { app: "uploader", uid: "***LOGIN***", level: "2", error: {…}, file: File, upload: {…} }
ConsoleLogger.js:74:18
[DEBUG] uploader: Successfully uploaded 10 
Object { app: "uploader", uid: "***LOGIN***", level: "2", file: File, upload: {…} }
ConsoleLogger.js:65:18
XHRPUT
https://***URL-PLATEFORME***/remote.php/dav/files/***LOGIN***/testDrag/6
[HTTP/1.1 403 Forbidden 28ms]

[ERROR] uploader: Failed uploading 6 
Object { app: "uploader", uid: "***LOGIN***", level: "2", error: {…}, file: File, upload: {…} }
ConsoleLogger.js:74:18
[DEBUG] uploader: Successfully uploaded 1 
Object { app: "uploader", uid: "***LOGIN***", level: "2", file: File, upload: {…} }
ConsoleLogger.js:65:18
XHRPUT
https://***URL-PLATEFORME***/remote.php/dav/files/***LOGIN***/testDrag/5
[HTTP/1.1 403 Forbidden 32ms]

[DEBUG] uploader: Successfully uploaded 9 
Object { app: "uploader", uid: "***LOGIN***", level: "2", file: File, upload: {…} }
ConsoleLogger.js:65:18
XHRPUT
https://***URL-PLATEFORME***/remote.php/dav/files/***LOGIN***/testDrag/8
[HTTP/1.1 403 Forbidden 25ms]

[ERROR] uploader: Failed uploading 5 
Object { app: "uploader", uid: "***LOGIN***", level: "2", error: {…}, file: File, upload: {…} }
ConsoleLogger.js:74:18
[ERROR] uploader: Failed uploading 8 
Object { app: "uploader", uid: "***LOGIN***", level: "2", error: {…}, file: File, upload: {…} }
ConsoleLogger.js:74:18
[DEBUG] uploader: Successfully uploaded 7 
Object { app: "uploader", uid: "***LOGIN***", level: "2", file: File, upload: {…} }
ConsoleLogger.js:65:18
[ERROR] files: Error while uploading files 
Object { app: "files", uid: "***LOGIN***", level: "2", errors: (6) […] }
ConsoleLogger.js:74:18
[DEBUG] files: VirtualList: resizeObserver updated 
Object { app: "files", uid: "***LOGIN***", level: "2" }
ConsoleLogger.js:65:18
[DEBUG] files: VirtualList: resizeObserver updated 
Object { app: "files", uid: "***LOGIN***", level: "2" }
ConsoleLogger.js:65:18
[DEBUG] files: VirtualList: resizeObserver updated 
Object { app: "files", uid: "***LOGIN***", level: "2" }
ConsoleLogger.js:65:18
[DEBUG] files: VirtualList: resizeObserver updated 
Object { app: "files", uid: "***LOGIN***", level: "2" }
ConsoleLogger.js:65:18

Apache2 first node:

192.168.1.1 - - [27/Jun/2024:12:22:51 +0200] "PROPFIND /remote.php/dav/files/***LOGIN***/testDrag HTTP/1.1" 404 973 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:127.0) Gecko/20100101 Firefox/127.0"
192.168.1.1 - - [27/Jun/2024:12:22:51 +0200] "PROPFIND /remote.php/dav/files/***LOGIN*** HTTP/1.1" 207 1178 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:127.0) Gecko/20100101 Firefox/127.0"
192.168.1.1 - - [27/Jun/2024:12:22:52 +0200] "PROPFIND /remote.php/dav/files/***LOGIN***/testDrag HTTP/1.1" 207 1498 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:127.0) Gecko/20100101 Firefox/127.0"
192.168.1.1 - - [27/Jun/2024:12:22:52 +0200] "PUT /remote.php/dav/files/***LOGIN***/testDrag/4 HTTP/1.1" 403 736 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:127.0) Gecko/20100101 Firefox/127.0"
192.168.1.1 - - [27/Jun/2024:12:22:52 +0200] "PUT /remote.php/dav/files/***LOGIN***/testDrag/3 HTTP/1.1" 403 736 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:127.0) Gecko/20100101 Firefox/127.0"
192.168.1.1 - - [27/Jun/2024:12:22:52 +0200] "PUT /remote.php/dav/files/***LOGIN***/testDrag/9 HTTP/1.1" 403 736 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:127.0) Gecko/20100101 Firefox/127.0"
192.168.1.1 - - [27/Jun/2024:12:22:52 +0200] "PUT /remote.php/dav/files/***LOGIN***/testDrag/6 HTTP/1.1" 403 736 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:127.0) Gecko/20100101 Firefox/127.0"
192.168.1.1 - - [27/Jun/2024:12:22:52 +0200] "PUT /remote.php/dav/files/***LOGIN***/testDrag/8 HTTP/1.1" 403 736 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:127.0) Gecko/20100101 Firefox/127.0"

Apache2 second node:

192.168.1.1 - - [27/Jun/2024:12:22:51 +0200] "PROPFIND /remote.php/dav/files/***LOGIN***/ HTTP/1.1" 207 2249 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:127.0) Gecko/20100101 Firefox/127.0"
192.168.1.1 - - [27/Jun/2024:12:22:51 +0200] "PROPFIND /remote.php/dav/files HTTP/1.1" 207 1056 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:127.0) Gecko/20100101 Firefox/127.0"
192.168.1.1 - - [27/Jun/2024:12:22:52 +0200] "PROPFIND /remote.php/dav/files/***LOGIN***/testDrag HTTP/1.1" 207 1180 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:127.0) Gecko/20100101 Firefox/127.0"
192.168.1.1 - - [27/Jun/2024:12:22:52 +0200] "GET /index.php/apps/files/api/v1/stats HTTP/1.1" 200 973 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:127.0) Gecko/20100101 Firefox/127.0"
192.168.1.1 - - [27/Jun/2024:12:22:52 +0200] "PUT /remote.php/dav/files/***LOGIN***/testDrag/10 HTTP/1.1" 204 686 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:127.0) Gecko/20100101 Firefox/127.0"
192.168.1.1 - - [27/Jun/2024:12:22:52 +0200] "PUT /remote.php/dav/files/***LOGIN***/testDrag/2 HTTP/1.1" 201 742 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:127.0) Gecko/20100101 Firefox/127.0"
192.168.1.1 - - [27/Jun/2024:12:22:52 +0200] "PUT /remote.php/dav/files/***LOGIN***/testDrag/1 HTTP/1.1" 204 686 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:127.0) Gecko/20100101 Firefox/127.0"
192.168.1.1 - - [27/Jun/2024:12:22:52 +0200] "PUT /remote.php/dav/files/***LOGIN***/testDrag/7 HTTP/1.1" 204 686 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:127.0) Gecko/20100101 Firefox/127.0"
192.168.1.1 - - [27/Jun/2024:12:22:52 +0200] "PUT /remote.php/dav/files/***LOGIN***/testDrag/5 HTTP/1.1" 201 742 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:127.0) Gecko/20100101 Firefox/127.0"
192.168.1.1 - - [27/Jun/2024:12:22:57 +0200] "GET /ocs/v2.php/apps/notifications/api/v2/notifications HTTP/1.1" 304 758 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:127.0) Gecko/20100101 Firefox/127.0"
dsisysteme commented 1 month ago

The issue is still present in 29.0.4

skjnldsv commented 1 month ago

Hi,

Can you share the exact 403 error from the logs please? I'm guessing your two instances share the same DB and data folder?

dsisysteme commented 1 month ago

Hi,

Could you specify the information you are missing regarding the apache2 logs we included in the issue description?

Regarding the second question, indeed, the two instances share the same database and the same data folder. We also have a Redis instance that stores PHP sessions and distributed cache.

skjnldsv commented 1 month ago

I meant the content of the 403 response. Either the json entry in nextcloud.log if any, or the browser network request response.

skjnldsv commented 1 month ago

On a slight different note, can you share how you configured/enabled your shared session ?

dsisysteme commented 1 month ago

Unfortunately, there is no log in the nextcloud.log. Here is an image of the sequence of requests: image

All PUT requests with a 201 status are handled by node 2, and the 403 errors are on node 1. It seems that the node which handled the MKCOL request is the one where the PUT requests have a 201 status.

here is the extract of the responses from the requests at the browser level: PUT 403:

HTTP/1.1 403 Forbidden
Date: Mon, 29 Jul 2024 13:37:03 GMT
Server: Apache
Strict-Transport-Security: max-age=15768000
Referrer-Policy: no-referrer
X-Content-Type-Options: nosniff
X-Frame-Options: SAMEORIGIN
X-Permitted-Cross-Domain-Policies: none
X-Robots-Tag: noindex, nofollow
X-XSS-Protection: 1; mode=block
Content-Security-Policy: default-src 'none';
Content-Type: application/xml; charset=utf-8
Keep-Alive: timeout=5, max=92
Connection: Keep-Alive
Transfer-Encoding: chunked

PUT 201:

HTTP/1.1 201 Created
Date: Mon, 29 Jul 2024 13:37:03 GMT
Server: Apache
Strict-Transport-Security: max-age=15768000
Referrer-Policy: no-referrer
X-Content-Type-Options: nosniff
X-Frame-Options: SAMEORIGIN
X-Permitted-Cross-Domain-Policies: none
X-Robots-Tag: noindex, nofollow
X-XSS-Protection: 1; mode=block
Content-Security-Policy: default-src 'none';
X-OC-MTime: accepted
OC-FileId: 250391672oc7elpk9mous
X-Request-Id: rz2VVlqVxSm4UqXra3xI
OC-ETag: "c1677a26a6ae01fc281bb0a4905400a3"
X-Debug-Token: rz2VVlqVxSm4UqXra3xI
ETag: "c1677a26a6ae01fc281bb0a4905400a3"
Content-Length: 0
Content-Type: text/html; charset=UTF-8
Keep-Alive: timeout=5, max=100
Connection: Keep-Alive

For PHP sessions, we use the following configuration at the fpm pool level on the 2 nodes:

php_admin_value[session.save_path] = tcp://<redis-server>:6379
php_admin_value[session.save_handler] = redis

For nextcloud, we force the distributed cache to index 1 to avoid conflicts. "dbindex": 1

skjnldsv commented 1 month ago

here is the extract of the responses from the requests at the browser level: PUT 403:

Please also give us the xml body response, not only the headers

dsisysteme commented 1 month ago

Sorry for the misunderstanding.

Here is the XML body of the 403 response:

<?xml version="1.0" encoding="utf-8"?>
<d:error xmlns:d="DAV:" xmlns:s="http://sabredav.org/ns">
  <s:exception>Sabre\DAV\Exception\Forbidden</s:exception>
  <s:message/>
</d:error>
dsisysteme commented 1 week ago

The issue is still present in 29.0.5