SMB External storage webdav path adds extra string "Dc" and causes share to be not accessible

uniuuu commented 2 years ago

How to use GitHub

Please use the 👍 reaction to show that you are affected by the same issue.
Please don't comment if you have no relevant information to add. It's just extra noise for everyone subscribed to this issue.
Subscribe to receive notifications on status change and new comments.

Steps to reproduce

To have installation with external storage using SMB share.
To upgrade from 21.0.4 to 22.1.1
Go to external storage mounted folder and see nested folder names changed adding "Dc" at the end

Expected behaviour

External storage will nested folders names not changed

Actual behaviour

External storage nested folders names have changed adding "Dc" at the end

Server configuration

Operating system: Debian (in docker)

Web server: Apache

Database: MySQL

PHP version: 8.0.11

Nextcloud version: (see Nextcloud admin page) 22.1.1

Updated from an older Nextcloud/ownCloud or fresh install: 21.0.4

Where did you install Nextcloud from: docker

Signing status:

Signing status

``` No errors have been found. ```

List of activated apps:

App list

``` Enabled: - activity: 2.15.0 - admin_audit: 1.11.0 - cloud_federation_api: 1.4.0 - dav: 1.18.0 - federatedfilesharing: 1.11.0 - files: 1.16.0 - files_external: 1.12.1 - files_sharing: 1.13.2 - files_trashbin: 1.11.0 - impersonate: 1.9.0 - logreader: 2.7.0 - lookup_server_connector: 1.9.0 - oauth2: 1.9.0 - password_policy: 1.12.0 - provisioning_api: 1.11.0 - settings: 1.3.0 - twofactor_admin: 3.1.0 - twofactor_backupcodes: 1.10.1 - twofactor_totp: 6.1.0 - user_ldap: 1.11.0 - viewer: 1.6.0 - workflowengine: 2.3.1 Disabled: - accessibility - analytics - announcementcenter - appointments - auto_groups - bookmarks - calendar - carnet - checksum - circles - cms_pico - comments - contacts - contactsinteraction - cospend - dashboard - data_request - deck - documentserver_community - drawio - duplicatefinder - encryption - event_update_notification - external - extract - facerecognition - federation - files_accesscontrol - files_antivirus - files_automatedtagging - files_fulltextsearch - files_mindmap - files_pdfviewer - files_photospheres - files_rightclick - files_trackdownloads - files_versions - files_videoplayer - firstrunwizard - forms - fulltextsearch - groupfolders - health - integration_suitecrm - integration_twitter - keeweb - ldap_write_support - login_notes - mail - maps - metadata - nextcloud_announcements - notes - notifications - ocdownloader - onlyoffice - passman - passwords - phonetrack - photos - polls - privacy - quicknotes - rainloop - ransomware_protection - recommendations - registration - richdocuments - richdocumentscode - riotchat - sendent - serverinfo - sharebymail - sharelisting - sharingpath - smb_test - spreed - support - survey_client - suspicious_login - systemtags - tasks - text - theming - theming_customcss - timemanager - timetracker - unsplash - updatenotification - user_status - video_converter - weather_status - workflow_ocr - workflow_pdf_converter - workflow_script ```

Nextcloud configuration:

Config report

``` { "system": { "htaccess.RewriteBase": "\/", "skeletondirectory": "", "memcache.local": "\\OC\\Memcache\\APCu", "memcache.locking": "\\OC\\Memcache\\Redis", "redis": { "host": "***REMOVED SENSITIVE VALUE***", "port": 6379 }, "apps_paths": [ { "path": "\/var\/www\/html\/apps", "url": "\/apps", "writable": false }, { "path": "\/var\/www\/html\/custom_apps", "url": "\/custom_apps", "writable": true } ], "instanceid": "***REMOVED SENSITIVE VALUE***", "passwordsalt": "***REMOVED SENSITIVE VALUE***", "secret": "***REMOVED SENSITIVE VALUE***", "trusted_domains": [ "***REMOVED SENSITIVE VALUE***", "172.25.200.128", "10.218.99.7" ], "trusted_proxies": "***REMOVED SENSITIVE VALUE***", "default_phone_region": "SG", "datadirectory": "***REMOVED SENSITIVE VALUE***", "dbtype": "mysql", "version": "22.1.1.2", "overwrite.cli.url": "***REMOVED SENSITIVE VALUE***", "dbname": "***REMOVED SENSITIVE VALUE***", "dbhost": "***REMOVED SENSITIVE VALUE***", "dbport": "", "dbtableprefix": "oc_", "mysql.utf8mb4": true, "dbuser": "***REMOVED SENSITIVE VALUE***", "dbpassword": "***REMOVED SENSITIVE VALUE***", "installed": true, "ldapIgnoreNamingRules": false, "ldapProviderFactory": "OCA\\User_LDAP\\LDAPProviderFactory", "logtimezone": "Asia\/Singapore", "log_type": "owncloud", "logfile": "\/var\/log\/nextcloud.log", "loglevel": 2, "log_rotate_size": "104857600", "twofactor_enforced": "true", "twofactor_enforced_groups": [], "twofactor_enforced_excluded_groups": [ "Test" ], "maintenance": false, "mail_from_address": "***REMOVED SENSITIVE VALUE***", "mail_smtpmode": "smtp", "mail_sendmailmode": "smtp", "mail_domain": "***REMOVED SENSITIVE VALUE***", "mail_smtphost": "***REMOVED SENSITIVE VALUE***", "mail_smtpport": "465", "mail_smtpauth": 1, "mail_smtpauthtype": "LOGIN", "mail_smtpname": "***REMOVED SENSITIVE VALUE***", "mail_smtppassword": "***REMOVED SENSITIVE VALUE***", "app.mail.verify-tls-peer": false, "mail_smtpdebug": false, "defaultapp": "files", "app_install_overwrite": [ "spreedme", "twofactor_admin" ], "theme": "", "has_rebuilt_cache": true, "debug": false, "overwriteprotocol": "https", "mail_smtpsecure": "ssl", "session_lifetime": 2400, "remember_login_cookie_lifetime": 3600, "updater.secret": "***REMOVED SENSITIVE VALUE***" } } ```

Are you using external storage, if yes which one: SMB

Are you using encryption: no

Are you using an external user-backend, if yes which one: LDAP

LDAP configuration (delete this part if not used)

LDAP config

Client configuration

Browser: Firefox

Operating system: Windows 10

Logs

Web server error log

``` Insert your webserver log here ```

Nextcloud log (data/nextcloud.log)

Nextcloud log

``` {"reqId":"ah7FqsKlV6eTZeRHzlru","level":4,"time":"2021-09-30T10:07:04+08:00","remoteAddr":"58.182.68.120","user":"***REMOVED SENSITIVE VALUE*** ","app":"webdav","method":"GET","url":"/remote.php/webdav/TFS%20Data/2021%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20Dc","message":"Could not open file","userAgent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:92.0) Gecko/20100101 Firefox/92.0","version":"22.1.1.2","exception":{"Exception":"Sabre\\DAV\\Exception\\ServiceUnavailable","Message":"Could not open file","Code":0,"Trace":[{"file":"/var/www/html/3rdparty/sabre/dav/lib/DAV/CorePlugin.php","line":85,"function":"get","class":"OCA\\DAV\\Connector\\Sabre\\File","type":"->","args":[]},{"file":"/var/www/html/3rdparty/sabre/event/lib/WildcardEmitterTrait.php","line":89,"function":"httpGet","class":"Sabre\\DAV\\CorePlugin","type":"->","args":[{"__class__":"Sabre\\HTTP\\Request"},{"__class__":"Sabre\\HTTP\\Response"}]},{"file":"/var/www/html/3rdparty/sabre/dav/lib/DAV/Server.php","line":472,"function":"emit","class":"Sabre\\DAV\\Server","type":"->","args":["method:GET",[{"__class__":"Sabre\\HTTP\\Request"},{"__class__":"Sabre\\HTTP\\Response"}]]},{"file":"/var/www/html/3rdparty/sabre/dav/lib/DAV/Server.php","line":253,"function":"invokeMethod","class":"Sabre\\DAV\\Server","type":"->","args":[{"__class__":"Sabre\\HTTP\\Request"},{"__class__":"Sabre\\HTTP\\Response"}]},{"file":"/var/www/html/3rdparty/sabre/dav/lib/DAV/Server.php","line":321,"function":"start","class":"Sabre\\DAV\\Server","type":"->","args":[]},{"file":"/var/www/html/apps/dav/appinfo/v1/webdav.php","line":83,"function":"exec","class":"Sabre\\DAV\\Server","type":"->","args":[]},{"file":"/var/www/html/remote.php","line":166,"args":["/var/www/html/apps/dav/appinfo/v1/webdav.php"],"function":"require_once"}],"File":"/var/www/html/apps/dav/lib/Connector/Sabre/File.php","Line":438,"CustomMessage":"--"},"id":"61551bd263ecf"} ```

Browser log

``` Insert your browser log here, this could for example include: https://***REMOVED SENSITIVE VALUE***/remote.php/webdav/TFS%20Data/2021%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20Dc HTTP/2 503 Service Unavailable cache-control: no-store, no-cache, must-revalidate content-security-policy: default-src 'none';base-uri 'none';manifest-src 'self';script-src 'self';style-src 'self' 'unsafe-inline';img-src 'self' data: blob:;font-src 'self' data:;connect-src 'self';media-src 'self';frame-ancestors 'self';form-action 'self' content-type: text/html; charset=UTF-8 date: Thu, 30 Sep 2021 02:10:14 GMT expires: Thu, 19 Nov 1981 08:52:00 GMT pragma: no-cache referrer-policy: no-referrer server: Apache/2.4.48 (Debian) strict-transport-security: max-age=31536000 x-content-type-options: nosniff x-download-options: noopen x-frame-options: SAMEORIGIN x-permitted-cross-domain-policies: none x-powered-by: PHP/8.0.11 x-robots-tag: none x-xss-protection: 1; mode=block X-Firefox-Spdy: h2 ```

uniuuu commented 2 years ago

See the screencast attached

https://user-images.githubusercontent.com/26146469/135376001-1cb831a7-4c65-402b-a750-21bcf7aca9a6.mp4

uniuuu commented 2 years ago

The main point that the path changed. It has to be https://sample.com/remote.php/webdav/TFS%20Data/2021 But it comes with spaces and Dc at the end. https://sample.com/remote.php/webdav/TFS%20Data/2021%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20Dc

In mobile application the same impact: WhatsApp Image 2021-09-29 at 1 56 07 PM

No issue found on source file server. file-server

solracsf commented 2 years ago

Do you confirm the same issue with 22.2.0?

NA-NOOK commented 2 years ago

I can confirm this bug on NC 20.0.12, the problem is caused by an incorrect interpretation of file attributes, see this thread I started recently. In short: this "extra string" is filename attribute merged with the file name. In this context "Dc" suffix means that directory and compressed attribute is set on smb share.

ghost commented 2 years ago

This issue has been automatically marked as stale because it has not had recent activity and seems to be missing some essential information. It will be closed if no further activity occurs. Thank you for your contributions.

SteffenDE commented 2 years ago

I can confirm the same issue using 22.2.0 with directories showing as files ending with "Dr":

smbclient listing: tg_image_753850213

SteffenDE commented 2 years ago

It works when using the native smbclient (https://github.com/eduardok/libsmbclient-php). After purging the external storages from the database, the directories are fine again for me. This is my new Dockerfile:

FROM nextcloud:22-fpm

RUN apt-get update && apt-get install -y procps libsmbclient-dev && rm -rf /var/lib/apt/lists/*
RUN pecl install smbclient && docker-php-ext-enable smbclient

It can also be fixed by adding lowercase modes to the regex in apps/files_external/3rdparty/icewind/smb/src/Wrapped/Parser.php and adding a strtoupper:

        /**
     * @param string[] $output
     * @param string $basePath
     * @param callable(string):ACL[] $aclCallback
     * @return FileInfo[]
     */
    public function parseDir(array $output, string $basePath, callable $aclCallback): array {
        //last line is used space
        array_pop($output);
        $regex = '/^\s*(.*?)\s\s\s\s+(?:([NDHARSndhars]*)\s+)?([0-9]+)\s+(.*)$/';
        //2 spaces, filename, optional type, size, date
        $content = [];
        foreach ($output as $line) {
            if (preg_match($regex, $line, $matches)) {
                list(, $name, $mode, $size, $time) = $matches;
                if ($name !== '.' and $name !== '..') {
                    $mode = $this->parseMode(strtoupper($mode));
                    $time = strtotime($time . ' ' . $this->timeZone);
                    $path = $basePath . '/' . $name;
                    $content[] = new FileInfo($path, $name, (int)$size, $time, $mode, function () use ($aclCallback, $path): array {
                        return $aclCallback($path);
                    });
                }
            }
        }
        return $content;
    }

ghost commented 2 years ago

This issue has been automatically marked as stale because it has not had recent activity and seems to be missing some essential information. It will be closed if no further activity occurs. Thank you for your contributions.

uniuuu commented 2 years ago

Do you confirm the same issue with 22.2.0?

Hi @acsfer

On the Nextcloud Hub II (23.0.0) the issue is still persisting.

solracsf commented 2 years ago

@uniuuu can you try the patch by https://github.com/nextcloud/server/issues/29003#issuecomment-962649951 (modify the file apps/files_external/3rdparty/icewind/smb/src/Wrapped/Parser.php)

uniuuu commented 2 years ago

@uniuuu can you try the patch by #29003 (comment) (modify the file apps/files_external/3rdparty/icewind/smb/src/Wrapped/Parser.php)

Thank you. Tried change Dockerfile instead and it worked out by removing smbclient deb package and installing as per below. RUN pecl install smbclient && docker-php-ext-enable smbclient I confirm the folder's content started to show.

solracsf commented 2 years ago

Closing and moving issue + possible fix to https://github.com/icewind1991/SMB/issues/110

nextcloud / server