[Bug]: External SMB share display cache instead of real files, unusable

huntervcx commented 4 months ago

⚠️ This issue respects the following points: ⚠️

[X] This is a bug, not a question or a configuration/webserver/proxy issue.
[X] This issue is not already reported on Github OR Nextcloud Community Forum (I've searched it).
[X] Nextcloud Server is up to date. See Maintenance and Release Schedule for supported versions.
[X] I agree to follow Nextcloud's Code of Conduct.

Bug description

Nextcloud Server with multiple windows smb shares. Some folders use nextcloud cache to display folders and files, even files that have been deleted and new ones not displayed on nextcloud No update possible, multiple commands used without effect Share is used with cached credentials saved in database, so notify is not possible and scan can't be done from cli

~600 shares configured, randomly breaking some. But some folders don't, and continue to work properly Login/logout, disabling redis cache, restarting, cleanup files, trying to upload and delete from NC donesn't retrigger a cache refresh

Not a permissions problem, this is random, and even with full permissions and admins accounts, still displaying cache folders list

Steps to reproduce

Add an external file smb share
Work on some files directly on the smb share
Wait for nextcloud to Randomly stop reading from backend but only from cache

Expected behavior

Displaying correct folders and files, cleaning up the database filecache

Installation method

Community Manual installation with Archive

Nextcloud Server version

29

Operating system

Debian/Ubuntu

PHP engine version

PHP 8.2

Web server

Apache (supported)

Database engine version

MariaDB

Is this bug present after an update or on a fresh install?

Upgraded to a MAJOR version (ex. 22 to 23)

Are you using the Nextcloud Server Encryption module?

None

What user-backends are you using?

[ ] Default user-backend (database)
[X] LDAP/ Active Directory
[ ] SSO - SAML
[ ] Other

Configuration report

{
    "system": {
        "instanceid": "***REMOVED SENSITIVE VALUE***",
        "passwordsalt": "***REMOVED SENSITIVE VALUE***",
        "secret": "***REMOVED SENSITIVE VALUE***",
        "trusted_domains": [
            "***REMOVED SENSITIVE VALUE***",
        ],
        "trusted_proxies": "***REMOVED SENSITIVE VALUE***",
        "datadirectory": "***REMOVED SENSITIVE VALUE***",
        "dbtype": "mysql",
        "version": "29.0.0.19",
        "overwrite.cli.url": "***REMOVED SENSITIVE VALUE***",
        "dbname": "***REMOVED SENSITIVE VALUE***",
        "dbhost": "***REMOVED SENSITIVE VALUE***",
        "dbport": "",
        "dbtableprefix": "oc_",
        "mysql.utf8mb4": true,
        "dbuser": "***REMOVED SENSITIVE VALUE***",
        "dbpassword": "***REMOVED SENSITIVE VALUE***",
        "installed": true,
        "htaccess.RewriteBase": "\/",
        "ldapProviderFactory": "OCA\\User_LDAP\\LDAPProviderFactory",
        "mail_smtpmode": "smtp",
        "mail_smtpauth": 1,
        "mail_sendmailmode": "smtp",
        "mail_from_address": "***REMOVED SENSITIVE VALUE***",
        "mail_domain": "***REMOVED SENSITIVE VALUE***",
        "mail_smtpauthtype": "LOGIN",
        "mail_smtphost": "***REMOVED SENSITIVE VALUE***",
        "mail_smtpname": "***REMOVED SENSITIVE VALUE***",
        "mail_smtppassword": "***REMOVED SENSITIVE VALUE***",
        "mail_smtpport": "465",
        "skeletondirectory": "",
        "templatedirectory": "",
        "mail_smtpsecure": "ssl",
        "default_phone_region": "fr_FR",
        "default_language": "fr",
        "default_locale": "fr_FR",
        "trashbin_retention_obligation": "15,90",
        "knowledgebaseenabled": false,
        "maintenance": false,
        "enable_previews": true,
        "memcache.local": "\\OC\\Memcache\\Redis",
        "memcache.locking": "\\OC\\Memcache\\Redis",
        "filelocking.enabled": "true",
        "redis": {
            "host": "***REMOVED SENSITIVE VALUE***",
            "port": 0,
            "timeout": 0
        },
        "loglevel": 2,
        "maintenance_window_start": 2,
        "activity_use_cached_mountpoints": true,
        "log_type": "file",
        "logfile": "***REMOVED SENSITIVE VALUE***",
        "logdateformat": "F d, Y H:i:s"
    }
}

List of activated Apps

Enabled:
  - activity: 2.21.1
  - admin_audit: 1.19.0
  - bruteforcesettings: 2.9.0
  - cloud_federation_api: 1.12.0
  - contactsinteraction: 1.10.0
  - dav: 1.30.1
  - external: 5.4.0
  - federatedfilesharing: 1.19.0
  - files: 2.1.0
  - files_accesscontrol: 1.19.0
  - files_antivirus: 5.5.0
  - files_downloadlimit: 2.0.0
  - files_external: 1.21.0
  - files_pdfviewer: 2.10.0
  - files_reminders: 1.2.0
  - files_sharing: 1.21.0
  - files_trashbin: 1.19.0
  - files_versions: 1.22.0
  - logreader: 2.14.0
  - lookup_server_connector: 1.17.0
  - notifications: 2.17.0
  - oauth2: 1.17.0
  - password_policy: 1.19.0
  - previewgenerator: 5.5.0
  - privacy: 1.13.0
  - provisioning_api: 1.19.0
  - recommendations: 2.1.0
  - serverinfo: 1.19.0
  - settings: 1.12.0
  - sharebymail: 1.19.0
  - theming: 2.4.0
  - twofactor_backupcodes: 1.18.0
  - twofactor_totp: 11.0.0-dev
  - updatenotification: 1.19.1
  - user_ldap: 1.20.0
  - viewer: 2.3.0
  - workflowengine: 2.11.0
Disabled:
  - circles: 29.0.0-dev (installed 25.0.0)
  - comments: 1.19.0 (installed 1.15.0)
  - dashboard: 7.9.0 (installed 7.5.0)
  - encryption: 2.17.0
  - federation: 1.19.0 (installed 1.15.0)
  - firstrunwizard: 2.18.0 (installed 2.14.0)
  - groupfolders: 16.0.4 (installed 16.0.4)
  - nextcloud_announcements: 1.18.0 (installed 1.14.0)
  - photos: 2.5.0 (installed 2.0.1)
  - related_resources: 1.4.0 (installed 1.3.0)
  - support: 1.12.0 (installed 1.8.0)
  - survey_client: 1.17.0 (installed 1.13.0)
  - suspicious_login: 7.0.0 (installed 6.0.0)
  - systemtags: 1.19.0 (installed 1.15.0)
  - text: 3.10.0 (installed 3.6.0)
  - user_status: 1.9.0 (installed 1.5.0)
  - weather_status: 1.9.0 (installed 1.5.0)

Nextcloud Signing status

No errors have been found.

Nextcloud Logs

{"reqId":"RSkVLYiLK9PdDLAEBcNR","level":3,"time":"May 08, 2024 22:53:04","remoteAddr":"1.2.3.4","user":"myadmin","app":"no app in context","method":"GET","url":"/ocs/v2.php/apps/dashboard/api/v1/widgets","message":"Class \"OCA\\Dashboard\\Controller\\DashboardController\" does not exist","userAgent":"Mozilla/5.0 (iOS) Nextcloud-iOS/5.2.7","version":"29.0.0.19","exception":{"Exception":"ReflectionException","Message":"Class \"OCA\\Dashboard\\Controller\\DashboardController\" does not exist","Code":-1,"Trace":[{"file":"/mnt/data/nextcloud/lib/private/Route/Router.php","line":469,"function":"__construct","class":"ReflectionClass","type":"->"},{"file":"/mnt/data/nextcloud/lib/private/Route/Router.php","line":161,"function":"getAttributeRoutes","class":"OC\\Route\\Router","type":"->"},{"file":"/mnt/data/nextcloud/lib/private/Route/Router.php","line":286,"function":"loadRoutes","class":"OC\\Route\\Router","type":"->"},{"file":"/mnt/data/nextcloud/lib/private/Route/Router.php","line":330,"function":"findMatchingRoute","class":"OC\\Route\\Router","type":"->"},{"file":"/mnt/data/nextcloud/ocs/v1.php","line":66,"function":"match","class":"OC\\Route\\Router","type":"->"},{"file":"/mnt/data/nextcloud/ocs/v2.php","line":23,"args":["/mnt/data/nextcloud/ocs/v1.php"],"function":"require_once"}],"File":"/mnt/data/nextcloud/lib/private/Route/Router.php","Line":469,"message":"Class \"OCA\\Dashboard\\Controller\\DashboardController\" does not exist","exception":{},"CustomMessage":"Class \"OCA\\Dashboard\\Controller\\DashboardController\" does not exist"}}
{"reqId":"KbKqdSTSi7euiSG1IbVQ","level":3,"time":"May 08, 2024 22:53:04","remoteAddr":"1.2.3.4","user":"myadmin","app":"no app in context","method":"GET","url":"/ocs/v2.php/apps/user_status/api/v1/user_status","message":"Class \"OCA\\UserStatus\\Controller\\UserStatusController\" does not exist","userAgent":"Mozilla/5.0 (iOS) Nextcloud-iOS/5.2.7","version":"29.0.0.19","exception":{"Exception":"ReflectionException","Message":"Class \"OCA\\UserStatus\\Controller\\UserStatusController\" does not exist","Code":-1,"Trace":[{"file":"/mnt/data/nextcloud/lib/private/Route/Router.php","line":469,"function":"__construct","class":"ReflectionClass","type":"->"},{"file":"/mnt/data/nextcloud/lib/private/Route/Router.php","line":161,"function":"getAttributeRoutes","class":"OC\\Route\\Router","type":"->"},{"file":"/mnt/data/nextcloud/lib/private/Route/Router.php","line":286,"function":"loadRoutes","class":"OC\\Route\\Router","type":"->"},{"file":"/mnt/data/nextcloud/lib/private/Route/Router.php","line":330,"function":"findMatchingRoute","class":"OC\\Route\\Router","type":"->"},{"file":"/mnt/data/nextcloud/ocs/v1.php","line":66,"function":"match","class":"OC\\Route\\Router","type":"->"},{"file":"/mnt/data/nextcloud/ocs/v2.php","line":23,"args":["/mnt/data/nextcloud/ocs/v1.php"],"function":"require_once"}],"File":"/mnt/data/nextcloud/lib/private/Route/Router.php","Line":469,"message":"Class \"OCA\\UserStatus\\Controller\\UserStatusController\" does not exist","exception":{},"CustomMessage":"Class \"OCA\\UserStatus\\Controller\\UserStatusController\" does not exist"}}
{"reqId":"TNyWJXtsMt9pR5L3OxXE","level":3,"time":"May 08, 2024 22:53:10","remoteAddr":"80.247.145.124","user":"myadmin","app":"webdav","method":"GET","url":"/remote.php/dav/files/myadmin/FOLDERSHARE/SubFolder/file.pdf","message":"Impossible d'ouvrir le fichier","userAgent":"Mozilla/5.0 (iOS) Nextcloud-iOS/5.2.7","version":"29.0.0.19","exception":{"Exception":"Sabre\\DAV\\Exception\\ServiceUnavailable","Message":"Impossible d'ouvrir le fichier","Code":0,"Trace":[{"file":"/mnt/data/nextcloud/3rdparty/sabre/dav/lib/DAV/CorePlugin.php","line":85,"function":"get","class":"OCA\\DAV\\Connector\\Sabre\\File","type":"->"},{"file":"/mnt/data/nextcloud/3rdparty/sabre/event/lib/WildcardEmitterTrait.php","line":89,"function":"httpGet","class":"Sabre\\DAV\\CorePlugin","type":"->"},{"file":"/mnt/data/nextcloud/3rdparty/sabre/dav/lib/DAV/Server.php","line":472,"function":"emit","class":"Sabre\\DAV\\Server","type":"->"},{"file":"/mnt/data/nextcloud/3rdparty/sabre/dav/lib/DAV/Server.php","line":253,"function":"invokeMethod","class":"Sabre\\DAV\\Server","type":"->"},{"file":"/mnt/data/nextcloud/3rdparty/sabre/dav/lib/DAV/Server.php","line":321,"function":"start","class":"Sabre\\DAV\\Server","type":"->"},{"file":"/mnt/data/nextcloud/apps/dav/lib/Server.php","line":374,"function":"exec","class":"Sabre\\DAV\\Server","type":"->"},{"file":"/mnt/data/nextcloud/apps/dav/appinfo/v2/remote.php","line":35,"function":"exec","class":"OCA\\DAV\\Server","type":"->"},{"file":"/mnt/data/nextcloud/remote.php","line":172,"args":["/mnt/data/nextcloud/apps/dav/appinfo/v2/remote.php"],"function":"require_once"}],"File":"/mnt/data/nextcloud/apps/dav/lib/Connector/Sabre/File.php","Line":505,"message":"Impossible d'ouvrir le fichier","exception":{},"CustomMessage":"Impossible d'ouvrir le fichier"}}

Additional info

Might be related, but pretty old : https://github.com/nextcloud/server/issues/6395

huntervcx commented 4 months ago

Ended up by truncate all of oc_filecache table, resolving all issues.

huntervcx commented 3 months ago

Bit of an update :

Reinstalled a full nextcloud server (same setup, deb12+mariadb via archive file but no redis cache, no trans lock). Still using "User credentials, save in database"

Same behaviour after 5 days, some shares doesn't update the file list hiding lots of files present on the smb backend Tried with three different accounts with different permissions (ro, rw, full control), no changes. Seems to read only from oc_filecache database, leading to unusable functionality :/

Will might try on a test server to give credentials stored in config with full access to see if nextcloud update the cache in background

huntervcx commented 3 months ago

Okay, another weird behaviour : Copied one file to two different smb shares. Using same account that have access to both, the file immediatly showed up in one folder, but not in the other one

Tried on a test server with credentials saved in config, no updates from nextcloud, forced to do an "occ files_external:scan 123" to list the files correctly Edit : 3hours later, test file still not showed on nextcloud (multiple reboots of the server, logout login on multiple accounts having access)

huntervcx commented 3 months ago

On a share having issues before : Moved folder A (with files) into folder B Nextcloud still list folder A but empty, but doesn't show it inside folder B

Tried to upload / delete files on other shares with different ids, they don't show any problems. share with id 226 show bugs, folders 66/99/101/200/266/642 have no issues, so doesn't seems to be loop issue

Checked oc_filecache, files appears from shares that are working, but no entry for the files inside the buggy shares

joshtrichards commented 2 months ago

Some folders use nextcloud cache to display folders and files, even files that have been deleted and new ones not displayed on nextcloud

Technically we always use the file cache. It is Nextcloud's file database. The "cache" bit may be a bit of a misnomer.

External Storage mounts will always have some delay or require some extra steps (like adding files:scan runs to your host cron like documented here).

That's the nature of what happens if you integrate storage that bypasses Nextcloud and has changes made to it outside of Nextcloud.

huntervcx commented 2 months ago

Yes, that is part of the problem : i wanted to use the login setting "login credentials, save in database" to get benefit from he external ntfs permissions I had to switch to a login:password type and cron the files:scan periodically, losing the permissions system

But yeah, some folders never got any update in like weekes, being used everyday on nextcloud on different clients, that's the bug i'm reporting here. Some would be refreshed live, some never got that chance

nextcloud / server