[Bug]: renaming file by just changing case leads to strange results (on SMB storage and others)

alexanderharm commented 11 months ago

⚠️ This issue respects the following points: ⚠️

[X] This is a bug, not a question or a configuration/webserver/proxy issue.
[X] This issue is not already reported on Github OR Nextcloud Community Forum (I've searched it).
[X] Nextcloud Server is up to date. See Maintenance and Release Schedule for supported versions.
[X] I agree to follow Nextcloud's Code of Conduct.

Bug description

Renaming an object (here folder/directory) on external storage connected via SMB leads to strange results depending on the client.

WebGUI:

renaming 99_test to 99_TEST produces an error message saying that the name is already in use and that I should use a different name
HOWEVER: in the GUI one can see now both folders 99_test AND 99_TEST
if I create an object in 99_test it also shows in 99_TEST and vice versa
if I delete either of the folders then both are either deleted or one is deleted and the other dysfunctional (cannot create any files or folders inside it)
on the connected Nextcloud client on Windows the user can just see the original 99_test

Nextcloud client (Windows) with virtual files:

renaming 99_test to 99_TEST produces an error message saying that it failed caused by a protocol error
client sees folder as 99_test, sync status remains active
creating any objects in 99_test fails in client and WebGUI, folder is dysfunctional

Steps to reproduce

create folder on external storage connected via SMB
rename the folder by just changing the case of a letter

Expected behavior

renaming by just changing case is detected by WebGUI and Nextcloud client as illegal on SMB storage and no file operations are executed.

Installation method

Other Community project

Nextcloud Server version

27

Operating system

Debian/Ubuntu

PHP engine version

PHP 8.2

Web server

Nginx

Database engine version

PostgreSQL

Is this bug present after an update or on a fresh install?

None

Are you using the Nextcloud Server Encryption module?

None

What user-backends are you using?

[X] Default user-backend (database)
[ ] LDAP/ Active Directory
[ ] SSO - SAML
[ ] Other

Configuration report

{
    "system": {
        "memcache.local": "\\OC\\Memcache\\APCu",
        "memcache.distributed": "\\OC\\Memcache\\Redis",
        "filelocking.enabled": true,
        "memcache.locking": "\\OC\\Memcache\\Redis",
        "datadirectory": "***REMOVED SENSITIVE VALUE***",
        "instanceid": "***REMOVED SENSITIVE VALUE***",
        "passwordsalt": "***REMOVED SENSITIVE VALUE***",
        "secret": "***REMOVED SENSITIVE VALUE***",
        "trusted_domains": [
            "cloud.satzkiste.de"
        ],
        "dbtype": "pgsql",
        "version": "27.1.1.0",
        "overwrite.cli.url": "https:\/\/cloud.satzkiste.de",
        "dbname": "***REMOVED SENSITIVE VALUE***",
        "dbhost": "***REMOVED SENSITIVE VALUE***",
        "dbport": "",
        "dbtableprefix": "oc_",
        "mysql.utf8mb4": true,
        "dbuser": "***REMOVED SENSITIVE VALUE***",
        "dbpassword": "***REMOVED SENSITIVE VALUE***",
        "installed": true,
        "trusted_proxies": "***REMOVED SENSITIVE VALUE***",
        "redis": {
            "host": "***REMOVED SENSITIVE VALUE***",
            "port": 6379,
            "timeout": 1
        },
        "default_language": "de",
        "default_locale": "de_DE",
        "skeletondirectory": "",
        "default_phone_region": "de",
        "maintenance": false,
        "loglevel": 3,
        "enable_previews": false,
        "updater.release.channel": "stable",
        "upgrade.disable-web": true,
        "mail_smtpmode": "smtp",
        "mail_smtpsecure": "tls",
        "mail_sendmailmode": "smtp",
        "mail_from_address": "***REMOVED SENSITIVE VALUE***",
        "mail_domain": "***REMOVED SENSITIVE VALUE***",
        "mail_smtpauthtype": "LOGIN",
        "mail_smtpauth": 1,
        "mail_smtphost": "***REMOVED SENSITIVE VALUE***",
        "mail_smtpport": "587",
        "mail_smtpname": "***REMOVED SENSITIVE VALUE***",
        "mail_smtppassword": "***REMOVED SENSITIVE VALUE***",
        "twofactor_enforced": "false",
        "twofactor_enforced_groups": [
            "admin",
            "StoryEditor"
        ],
        "twofactor_enforced_excluded_groups": [],
        "app_install_overwrite": [
            "gpxmotion",
            "googledocsredirect",
            "admin_notifications",
            "dropit",
            "file_upload_notification",
            "flowupload",
            "integration_jira",
            "cfg_share_links",
            "apporder",
            "facerecognition",
            "folderplayer"
        ],
        "memories.exiftool": "\/config\/www\/nextcloud\/apps\/memories\/exiftool-bin\/exiftool-amd64-musl",
        "memories.vod.path": "\/config\/www\/nextcloud\/apps\/memories\/exiftool-bin\/go-vod-amd64",
        "memories.vod.ffmpeg": "\/usr\/bin\/ffmpeg",
        "memories.vod.ffprobe": "\/usr\/bin\/ffprobe"
    }
}

List of activated Apps

Enabled:
  - activity: 2.19.0
  - admin_audit: 1.17.0
  - announcementcenter: 6.6.2
  - apporder: 0.15.0
  - bruteforcesettings: 2.7.0
  - cfg_share_links: 4.1.0
  - cloud_federation_api: 1.10.0
  - comments: 1.17.0
  - dashboard: 7.7.0
  - dav: 1.27.0
  - deck: 1.11.0
  - external: 5.2.1
  - externalportal: 1.2.0
  - federatedfilesharing: 1.17.0
  - federation: 1.17.0
  - files: 1.22.0
  - files_external: 1.19.0
  - files_pdfviewer: 2.8.0
  - files_reminders: 1.0.0
  - files_rightclick: 1.6.0
  - files_sharing: 1.19.0
  - files_trashbin: 1.17.0
  - files_versions: 1.20.0
  - groupfolders: 15.3.1
  - impersonate: 1.14.0
  - integration_google: 2.0.2
  - integration_openai: 1.0.13
  - logreader: 2.12.0
  - lookup_server_connector: 1.15.0
  - memories: 5.4.1
  - nextcloud_announcements: 1.16.0
  - notes: 4.8.1
  - notifications: 2.15.0
  - notify_push: 0.6.3
  - oauth2: 1.15.1
  - onlyoffice: 8.2.0
  - password_policy: 1.17.0
  - polls: 5.3.2
  - previewgenerator: 5.3.0
  - privacy: 1.11.0
  - provisioning_api: 1.17.0
  - quota_warning: 1.17.0
  - recommendations: 1.6.0
  - related_resources: 1.2.0
  - serverinfo: 1.17.0
  - settings: 1.9.0
  - sharebymail: 1.17.0
  - socialsharing_email: 2.6.0
  - spreed: 17.1.1
  - support: 1.10.0
  - survey_client: 1.15.0
  - suspicious_login: 5.0.0
  - systemtags: 1.17.0
  - text: 3.8.0
  - theming: 2.2.0
  - twofactor_backupcodes: 1.16.0
  - twofactor_totp: 9.0.0
  - unsplash: 2.2.1
  - user_status: 1.7.0
  - viewer: 2.1.0
  - weather_status: 1.7.0
  - welcome: 1.0.10
  - workflowengine: 2.9.0
Disabled:
  - appointments: 1.15.3 (installed 1.15.3)
  - calendar: 4.5.0 (installed 4.5.0)
  - calendar_resource_management: 0.5.0 (installed 0.5.0)
  - circles: 27.0.1 (installed 26.0.0)
  - contacts: 5.4.0 (installed 5.4.0)
  - contactsinteraction: 1.8.0 (installed 1.6.0)
  - encryption: 2.15.0
  - facerecognition: 0.9.31 (installed 0.9.31)
  - file_upload_notification: 0.1.2 (installed 0.1.2)
  - files_downloadlimit: 1.1.0 (installed 1.1.0)
  - firstrunwizard: 2.16.0 (installed 2.15.0)
  - flowupload: 1.1.3 (installed 1.1.3)
  - googledocsredirect: 1.0.2 (installed 1.0.2)
  - holiday_calendars: 0.3.0 (installed 0.3.0)
  - imageconverter: 1.3.5 (installed 1.3.5)
  - mailnotifier: 0.0.3 (installed 0.0.3)
  - maps: 1.1.1 (installed 1.1.1)
  - photos: 2.3.0 (installed 2.2.0)
  - ransomware_protection: 1.14.0 (installed 1.14.0)
  - recognize: 4.3.2 (installed 4.3.2)
  - richdocuments: 8.2.0 (installed 8.2.0)
  - snowflakestheme: 1.1.0 (installed 1.1.0)
  - unroundedcorners: 1.1.1 (installed 1.1.1)
  - user_ldap: 1.17.0
  - user_saml: 5.2.2 (installed 5.2.2)
  - workflow_media_converter: 1.8.4 (installed 1.8.4)

Nextcloud Signing status

No errors have been found.

Nextcloud Logs

{"reqId":"cn2Eniltym4LJ2uF0lYM","level":3,"time":"2023-09-28T19:08:51+00:00","remoteAddr":"142.132.133.228","user":"Maxi","app":"webdav","method":"MOVE","url":"/remote.php/dav/files/Maxi/EKS/99_test","message":"Failed to rename /EKS/99_test to /EKS/99_TEST","userAgent":"Mozilla/5.0 (Windows) mirall/3.10.0stable-Win64 (build 20230915) (Nextcloud, windows-10.0.19045 ClientArchitecture: x86_64 OsArchitecture: x86_64)","version":"27.1.1.0","exception":{"Exception":"Sabre\\DAV\\Exception","Message":"Failed to rename /EKS/99_test to /EKS/99_TEST","Code":0,"Trace":[{"file":"/app/www/public/3rdparty/sabre/dav/lib/DAV/Tree.php","line":153,"function":"setName","class":"OCA\\DAV\\Connector\\Sabre\\Node","type":"->"},{"file":"/app/www/public/3rdparty/sabre/dav/lib/DAV/CorePlugin.php","line":612,"function":"move","class":"Sabre\\DAV\\Tree","type":"->"},{"file":"/app/www/public/3rdparty/sabre/event/lib/WildcardEmitterTrait.php","line":89,"function":"httpMove","class":"Sabre\\DAV\\CorePlugin","type":"->"},{"file":"/app/www/public/3rdparty/sabre/dav/lib/DAV/Server.php","line":472,"function":"emit","class":"Sabre\\DAV\\Server","type":"->"},{"file":"/app/www/public/3rdparty/sabre/dav/lib/DAV/Server.php","line":253,"function":"invokeMethod","class":"Sabre\\DAV\\Server","type":"->"},{"file":"/app/www/public/3rdparty/sabre/dav/lib/DAV/Server.php","line":321,"function":"start","class":"Sabre\\DAV\\Server","type":"->"},{"file":"/config/www/nextcloud/apps/dav/lib/Server.php","line":365,"function":"exec","class":"Sabre\\DAV\\Server","type":"->"},{"file":"/config/www/nextcloud/apps/dav/appinfo/v2/remote.php","line":35,"function":"exec","class":"OCA\\DAV\\Server","type":"->"},{"file":"/app/www/public/remote.php","line":172,"args":["/config/www/nextcloud/apps/dav/appinfo/v2/remote.php"],"function":"require_once"}],"File":"/config/www/nextcloud/apps/dav/lib/Connector/Sabre/Node.php","Line":160,"message":"Failed to rename /EKS/99_test to /EKS/99_TEST","exception":[],"CustomMessage":"Failed to rename /EKS/99_test to /EKS/99_TEST"},"id":"6515d19e0186b"}

{"reqId":"nC8yKmehn3h4W7i6skb6","level":3,"time":"2023-09-28T18:46:04+00:00","remoteAddr":"87.138.252.179","user":"ncadmin","app":"no app in context","method":"POST","url":"/ocs/v2.php/apps/files/api/v1/templates/create","message":"Could not create path \"/ncadmin/files/EKS/99_test/test.md\"","userAgent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:109.0) Gecko/20100101 Firefox/115.0","version":"27.1.1.0","exception":{"Exception":"OCP\\Files\\NotPermittedException","Message":"Could not create path \"/ncadmin/files/EKS/99_test/test.md\"","Code":0,"Trace":[{"file":"/app/www/public/lib/private/Files/Template/TemplateManager.php","line":162,"function":"newFile","class":"OC\\Files\\Node\\Folder","type":"->"},{"file":"/config/www/nextcloud/apps/files/lib/Controller/TemplateController.php","line":57,"function":"createFromTemplate","class":"OC\\Files\\Template\\TemplateManager","type":"->","args":["*** sensitive parameters replaced ***"]},{"file":"/app/www/public/lib/private/AppFramework/Http/Dispatcher.php","line":230,"function":"create","class":"OCA\\Files\\Controller\\TemplateController","type":"->","args":["*** sensitive parameters replaced ***"]},{"file":"/app/www/public/lib/private/AppFramework/Http/Dispatcher.php","line":137,"function":"executeController","class":"OC\\AppFramework\\Http\\Dispatcher","type":"->"},{"file":"/app/www/public/lib/private/AppFramework/App.php","line":183,"function":"dispatch","class":"OC\\AppFramework\\Http\\Dispatcher","type":"->"},{"file":"/app/www/public/lib/private/Route/Router.php","line":315,"function":"main","class":"OC\\AppFramework\\App","type":"::"},{"file":"/app/www/public/ocs/v1.php","line":64,"function":"match","class":"OC\\Route\\Router","type":"->"},{"file":"/app/www/public/ocs/v2.php","line":23,"args":["/app/www/public/ocs/v1.php"],"function":"require_once"}],"File":"/app/www/public/lib/private/Files/Node/Folder.php","Line":193,"message":"Could not create path \"/ncadmin/files/EKS/99_test/test.md\"","exception":[],"CustomMessage":"Could not create path \"/ncadmin/files/EKS/99_test/test.md\""},"id":"6515d19e020df"}

Additional info

No response

joshtrichards commented 11 months ago

This may or may not be NC issue, but you'll need to dig a bit deeper into your SMB server's configuration and SMB implementation to help get to the bottom of the matter (your SMB server's underlying OS and OS version are a factor too).

It sounds like a bit like case sensitivity and/or case preservation is disabled on your SMB server (or any number of related options that depend on what your specific SMB server implementation supports).

Please check the manual for your SMB server to determine what it supports in terms of case preservation and sensitivity. You may just need to adjust some options. If it's really old (or oddly configured) there may even be some name mangling involved.

alexanderharm commented 11 months ago

Thanks for your feedback. The server runs Samba (Version 4.17.8-Debian) and is on default settings which according to Samba are the same for Windows Servers, meaning case insensitive but case preserving (https://www.samba.org/samba/docs/current/man-html/smb.conf.5.html) which would explain the above behaviour (but I would still suspect Nextcloud to handle the default config of SMB servers gracefully and not to end up with dysfunctional folders).

I can set case sensitive = yes for testing (only tonight) and I can also check if the dysfunctional folders in Nextcloud are actually dysfunctional on the SMB-server as well.

Rello commented 11 months ago

I can reproduce this error when using NC in docker and mounting the data volume

    volumes:
      - ./config:/var/www/html/config
      - ./data:/var/www/html/data

from nextcloud.log The destination node already exists, and the overwrite header is set to false

sorbaugh commented 11 months ago

cc @icewind1991 , @artonge

alexanderharm commented 11 months ago

I finally found some time to test further:

1. Test with Samba default settings

case sensitive = Auto

According to Samba manpage

The default setting of auto allows clients that support case sensitive filenames (Linux CIFSVFS and smbclient 3.0.5 and above currently) to tell the Samba server on a per-packet basis that they wish to access the file system in a case-sensitive manner (to support UNIX case sensitive semantics). No Windows or DOS system supports case-sensitive filename so setting this option to auto is that same as setting it to no for them.

Theoretically my server since based on Linux and using libsmbclient through a PHP-wrapper COULD use case sensitivity but doesn't
Renaming by changing case leads to above described issues

My test steps:

WebGUI

WebGUI: renamed folder from 99_test to 99_TEST
WebGUI: two folders appear namely 99_test and 99_TEST
WebGUI: files can be created in both and their content is always identical in the two folders
Nextcloud Client: only shows 99_test
SMB-mount: only shows 99_test
WebGUI: deleting 99_TEST leaves only 99_test but dysfunctional
Nextcloud Client: 99_test is dysfunctional
SMB-mount: 99_test and 99_TEST are both deleted
WebGUI: deleting 99_test generates error but deletes folder
Nextcloud Client: 99_test is deleted as well

Nextcloud Client

Nextcloud Client: renamed folder from 99_test to 99_TEST
Nextcloud Client: error message and 99_TEST shows but is dysfunctional
WebGUI: 99_TEST shows but is dysfunctional
SMB-mount: no folder anymore, it was deleted

SMB-mount

SMB-mount: renamed folder from 99_test to 99_TEST
Nextcloud Client: shows 99_TEST
WebGUI: 99_TEST shows

2. Test with case sensitivity

case sensitive = yes

Seems to work flawlessly

My test steps:

WebGUI: renamed folder from 99_test to 99_TEST
WebGUI: no error message and just 99_TEST
WebGUI: files can be created in the folder
Nextcloud Client: only shows 99_TEST
SMB-mount: only shows 99_TEST
SMB-mount: renamed folder from 99_TEST to 99_test
WebGUI: shows 99_test
Nextcloud Client: shows 99_test
Nextcloud Client: renaming works as well

3. Preliminary conclusion

Renaming folders on external storage by just changing case works in combination with Samba file servers if configured with case sensitive = yes
Did not have a Windows Server to test but AFAIK Windows does not support case sensitivity so the above issues might show again
I would still consider it a bug that the WebGUI ends up with two folders and if one is deleted the folder on the SMB share is deleted as well
The more serious bug probably lies in the Nextcloud client. Because renaming does lead directly to the folder being deleted on the share, the rename should abort before deleting data

e46lux commented 4 months ago

I feel like the issue lies with how the share is mounted via libsmbclient. I still haven't figured out how nextcloud specifies options to libsmbclient when mounting the share.

The following options:

vers=3.0,_netdev,credentials=/my_path/.secrets/.smb,iocharset=utf8,gid=100033,uid=100033,file_mode=0777,dir_mode=0777,sec=ntlmv2,cache=none,noserverino

allow linux to do a 'mv t.txt T.txt' without a problem.

Using: vers=2.0,_netdev,credentials=/my_path/.secrets/.smb,iocharset=utf8,gid=100033,uid=100033,file_mode=0777,dir_mode=0777,sec=ntlmv2

I get this file name already exists error directly in linux terminal.

Some post I crossed when searching for the issue referenced cache being a problem and setting cache=none helped resolve that. Not quite sure if switching to vers=3.0 or the noserverio options have any effect, but came with the suggestion from the post I found online.

Can anyone point me in the right direction where I can try specifying the SMB mount options values for testing? I found references in files.js and './apps/files_external/3rdparty/icewind/smb/src/Native/NativeState.php' but can't quite determine how the libsmbclient has options set for the connection.

nextcloud / server