smarinier
commented
4 years ago This is probably due to OSX files (not sure). Filenames in CACHE are stored with correct accentuated UTF-8 chars (like "Ć©" 195 169, LATIN SMALL LETTER E WITH ACUTE), but as the files are read, when Scanner enter in action to introduce new files, filenames are with combined UTF-8 ( e + 204 + 129). This mess up in Scanner.php (lib/private/Files/Cache/Scanner.php):
$removedChildren = \array_diff(array_keys($existingChildren), $newChildNames); where keys are in first UTF-8 sequence, and newChildNames in combined mode.
My proposal (work for me), is to "Normalize" UTF-8 filenames:
lib/private/Files/Cache/Scanner.php
419c419
< $newChildNames[] = $file;
---
> $newChildNames[] = \Normalizer::normalize($file);
kesselb
szaimen
How to use GitHub
Steps to reproduce
Expected behaviour
All files should be seen
Actual behaviour
Some files (with accentuated chars) do not appear on front side
Server configuration
Operating system: OSX
Web server: Apache
Database: SQLLite (default)
PHP version: 7.2.33
Nextcloud version: (see Nextcloud admin page)
Updated from an older Nextcloud/ownCloud or fresh install: Fresh install
Where did you install Nextcloud from: Tar file
Signing status:
Signing status
No errors have been found.List of activated apps:
App list
Enabled: - accessibility: 1.5.0 - activity: 2.12.0 - audioplayer: 2.11.2 - bruteforcesettings: 2.0.1 - camerarawpreviews: 0.7.8 - cloud_federation_api: 1.2.0 - comments: 1.9.0 - contactsinteraction: 1.0.0 - dav: 1.15.0 - drawio: 0.9.7 - external: 3.6.0 - federatedfilesharing: 1.9.0 - federation: 1.9.0 - files: 1.14.0 - files_external: 1.10.0 - files_mindmap: 0.0.22 - files_pdfviewer: 1.8.0 - files_rightclick: 0.16.0 - files_sharing: 1.11.0 - files_trashbin: 1.9.0 - files_versions: 1.12.0 - files_videoplayer: 1.8.0 - firstrunwizard: 2.8.0 - logreader: 2.4.0 - lookup_server_connector: 1.7.0 - music: 0.16.0 - nextcloud_announcements: 1.8.0 - notifications: 2.7.0 - oauth2: 1.7.0 - password_policy: 1.9.1 - photos: 1.1.0 - privacy: 1.3.0 - provisioning_api: 1.9.0 - recommendations: 0.7.0 - serverinfo: 1.9.0 - settings: 1.1.0 - sharebymail: 1.9.0 - support: 1.2.1 - survey_client: 1.7.0 - systemtags: 1.9.0 - text: 3.0.1 - theming: 1.10.0 - twofactor_backupcodes: 1.8.0 - updatenotification: 1.9.0 - viewer: 1.3.0 - workflowengine: 2.1.0 Disabled: - admin_audit - encryption - user_ldapNextcloud configuration:
Config report
{ "system": { "instanceid": "***REMOVED SENSITIVE VALUE***", "passwordsalt": "***REMOVED SENSITIVE VALUE***", "secret": "***REMOVED SENSITIVE VALUE***", "trusted_domains": [ "192.168.0.7" ], "datadirectory": "***REMOVED SENSITIVE VALUE***", "dbtype": "sqlite3", "version": "19.0.2.2", "overwrite.cli.url": "http:\/\/192.168.0.7\/nextcloud", "installed": true, "files_external_allow_create_new_local": "true", "logfile": "\/var\/log\/nexcloud.log", "mail_domain": "***REMOVED SENSITIVE VALUE***", "mail_from_address": "***REMOVED SENSITIVE VALUE***", "mail_smtpmode": "smtp", "mail_smtpauthtype": "LOGIN", "mail_smtphost": "***REMOVED SENSITIVE VALUE***", "mail_smtpport": "25", "app_install_overwrite": [ "radio" ], "maintenance": false } }Are you using external storage, if yes which one: local/smb/sftp/... Local Hard Drive Folders
Are you using encryption: yes/no no
Are you using an external user-backend, if yes which one: LDAP/ActiveDirectory/Webdav/... no
Client configuration
Browser: Chrome/Safari
Operating system: Mac OSX / iOS
Logs
Web server error log
Web server error log
No errors in apache log!Nextcloud log (data/nextcloud.log)
Nextcloud log
No significant logs ! {"reqId":"OeJECPK2e2EVGyqQT7iV","level":3,"time":"2020-09-10T19:36:16+00:00","remoteAddr":"192.168.0.11","user":"seb","app":"PHP","method":"POST","url":"/nextcloud/index.php/apps/files_external/globals torages","message":"stat(): stat failed for /Users/mini/Documents/Test2/ at /Users/mini/Sites/nextcloud/lib/private/Files/Storage/Local.php#148","userAgent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_1 3_4) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/84.0.4147.125 Safari/537.36","version":"19.0.2.2"} {"reqId":"BcBMMU4k7Zo8YCB8UG12","level":3,"time":"2020-09-10T19:36:27+00:00","remoteAddr":"192.168.0.11","user":"seb","app":"PHP","method":"PUT","url":"/nextcloud/index.php/apps/files_external/globalst orages/4","message":"stat(): stat failed for /Users/mini/Documents/Test/ at /Users/mini/Sites/nextcloud/lib/private/Files/Storage/Local.php#148","userAgent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_1 3_4) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/84.0.4147.125 Safari/537.36","version":"19.0.2.2"} {"reqId":"KWHsHBzmWPahOiMMhOpd","level":3,"time":"2020-09-10T19:36:37+00:00","remoteAddr":"192.168.0.11","user":"seb","app":"PHP","method":"GET","url":"/nextcloud/index.php/apps/files_external/globalst orages/4?testOnly=true","message":"stat(): stat failed for /Users/mini/Documents/Test/ at /Users/mini/Sites/nextcloud/lib/private/Files/Storage/Local.php#148","userAgent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_4) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/84.0.4147.125 Safari/537.36","version":"19.0.2.2"} {"reqId":"1Bukx9KZoUE9uid3F1eJ","level":3,"time":"2020-09-10T19:36:39+00:00","remoteAddr":"192.168.0.11","user":"seb","app":"PHP","method":"GET","url":"/nextcloud/index.php/apps/files_external/globalst orages/4?testOnly=true","message":"stat(): stat failed for /Users/mini/Documents/Test/ at /Users/mini/Sites/nextcloud/lib/private/Files/Storage/Local.php#148","userAgent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_4) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/84.0.4147.125 Safari/537.36","version":"19.0.2.2"} {"reqId":"8HABrZsyFfUqpjqoaLm6","level":3,"time":"2020-09-10T19:37:05+00:00","remoteAddr":"192.168.0.11","user":"seb","app":"PHP","method":"PUT","url":"/nextcloud/index.php/apps/files_external/globalst orages/4","message":"stat(): stat failed for /Users/mini/Documents/Test/ at /Users/mini/Sites/nextcloud/lib/private/Files/Storage/Local.php#148","userAgent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_1 3_4) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/84.0.4147.125 Safari/537.36","version":"19.0.2.2"} {"reqId":"EIkuXwsxOH0tT39z2TUa","level":3,"time":"2020-09-10T19:37:12+00:00","remoteAddr":"192.168.0.11","user":"seb","app":"PHP","method":"GET","url":"/nextcloud/index.php/apps/files_external/globalst orages/4?testOnly=true","message":"stat(): stat failed for /Users/mini/Documents/Test/ at /Users/mini/Sites/nextcloud/lib/private/Files/Storage/Local.php#148","userAgent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_4) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/84.0.4147.125 Safari/537.36","version":"19.0.2.2"} {"reqId":"tXsHeP5bJ1719pa1xph0","level":3,"time":"2020-09-10T19:37:32+00:00","remoteAddr":"192.168.0.11","user":"seb","app":"PHP","method":"PUT","url":"/nextcloud/index.php/apps/files_external/globalst orages/4","message":"stat(): stat failed for /Users/mini/Documents/Test/ at /Users/mini/Sites/nextcloud/lib/private/Files/Storage/Local.php#148","userAgent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_1 3_4) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/84.0.4147.125 Safari/537.36","version":"19.0.2.2"} {"reqId":"tndy64Jc7ttvLG2mORM3","level":3,"time":"2020-09-10T19:38:09+00:00","remoteAddr":"192.168.0.11","user":"seb","app":"PHP","method":"PUT","url":"/nextcloud/index.php/apps/files_external/globalst orages/4","message":"stat(): stat failed for /Users/mini/Documents/Test/ at /Users/mini/Sites/nextcloud/lib/private/Files/Storage/Local.php#148","userAgent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_1 3_4) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/84.0.4147.125 Safari/537.36","version":"19.0.2.2"} {"reqId":"oXeISeWqLrpE9NHURj6z","level":3,"time":"2020-09-10T19:38:10+00:00","remoteAddr":"192.168.0.11","user":"seb","app":"PHP","method":"PUT","url":"/nextcloud/index.php/apps/files_external/globalst orages/4","message":"stat(): stat failed for /Users/mini/Documents/Test/ at /Users/mini/Sites/nextcloud/lib/private/Files/Storage/Local.php#148","userAgent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_1 3_4) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/84.0.4147.125 Safari/537.36","version":"19.0.2.2"} {"reqId":"0jyBF1HZeroqvEW1pUDY","level":3,"time":"2020-09-10T19:38:11+00:00","remoteAddr":"192.168.0.11","user":"seb","app":"PHP","method":"PUT","url":"/nextcloud/index.php/apps/files_external/globalst orages/4","message":"stat(): stat failed for /Users/mini/Documents/Test/ at /Users/mini/Sites/nextcloud/lib/private/Files/Storage/Local.php#148","userAgent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_1 3_4) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/84.0.4147.125 Safari/537.36","version":"19.0.2.2"}Browser log
Browser log
Scanner Log:
Mac-mini-de-Mini:nextcloud mini$ sudo -u _www /usr/local/homebrew/bin/php occ files:scan --path=seb/files/MonTest Starting scan for user 1 out of 1 (seb) Entry "/seb/files/MonTest/C'est l'eĢteĢ deĢjaĢ.pdf" will not be accessible due to incompatible encoding Entry "/seb/files/MonTest/PreĢsentation 8.3.pdf" will not be accessible due to incompatible encoding +---------+-------+--------------+ | Folders | Files | Elapsed time | +---------+-------+--------------+ | 1 | 7 | 00:00:00 | +---------+-------+--------------+