nextcloud / server

☁️ Nextcloud server, a safe home for all your data
https://nextcloud.com
GNU Affero General Public License v3.0
27.44k stars 4.08k forks source link

File lock prevents file scan on external webdav storage #25596

Closed jfeldmaier closed 3 years ago

jfeldmaier commented 3 years ago

Steps to reproduce

  1. Nextcloud setup with external storage on IONOS HiDrive Storage (attached via webdav)
  2. Some files stored locally on the disk where the server is also installed
  3. Try to do a file scan for all users: sudo -u www-data php occ files:scan --all

Expected behaviour

The file scan should scan internal as well as the external files.

Actual behaviour

The file scan stops after scanning some of the locally stored files and reports an error on an external file:

Starting scan for user 1 out of 2 (xxx) Exception during scan: "files/93f47f2219cd30e07a85b622ab4xxxxxx"("webdav::someuser1234@webdav.hidrive.ionos.com///users/someuser1234/::scanner::") is locked

0 /var/www/html/cloud/lib/private/Files/Storage/Common.php(751): OC\Lock\DBLockingProvider->acquireLock()

1 /var/www/html/cloud/lib/private/Files/Storage/Wrapper/Wrapper.php(594): OC\Files\Storage\Common->acquireLock()

2 /var/www/html/cloud/lib/private/Files/Storage/Wrapper/Wrapper.php(594): OC\Files\Storage\Wrapper\Wrapper->acquireLock()

3 /var/www/html/cloud/lib/private/Files/Cache/Scanner.php(333): OC\Files\Storage\Wrapper\Wrapper->acquireLock()

4 /var/www/html/cloud/lib/private/Files/Utils/Scanner.php(260): OC\Files\Cache\Scanner->scan()

5 /var/www/html/cloud/apps/files/lib/Command/Scan.php(152): OC\Files\Utils\Scanner->scan()

6 /var/www/html/cloud/apps/files/lib/Command/Scan.php(208): OCA\Files\Command\Scan->scanFiles()

7 /var/www/html/cloud/apps/mail/vendor/symfony/console/Command/Command.php(258): OCA\Files\Command\Scan->execute()

8 /var/www/html/cloud/core/Command/Base.php(169): Symfony\Component\Console\Command\Command->run()

9 /var/www/html/cloud/apps/mail/vendor/symfony/console/Application.php(920): OC\Core\Command\Base->run()

10 /var/www/html/cloud/apps/mail/vendor/symfony/console/Application.php(266): Symfony\Component\Console\Application->doRunCommand()

11 /var/www/html/cloud/apps/mail/vendor/symfony/console/Application.php(142): Symfony\Component\Console\Application->doRun()

12 /var/www/html/cloud/lib/private/Console/Application.php(215): Symfony\Component\Console\Application->run()

13 /var/www/html/cloud/console.php(100): OC\Console\Application->run()

14 /var/www/html/cloud/occ(11): require_once('/var/www/html/c...')

15 {main}

Server configuration

Operating system: Linux 5.4.0-65-generic #73-Ubuntu SMP Mon Jan 18 17:25:17 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux

Web server: Server version: Apache/2.4.41 (Ubuntu) Server built: 2020-08-12T19:46:17

Database: Server version: 10.3.25-MariaDB-0ubuntu0.20.04.1 Ubuntu 20.04

PHP version: PHP 7.4.3 (cli) (built: Oct 6 2020 15:47:56) ( NTS ) Copyright (c) The PHP Group Zend Engine v3.4.0, Copyright (c) Zend Technologies with Zend OPcache v7.4.3, Copyright (c), by Zend Technologies

Nextcloud version: (see Nextcloud admin page) Nextcloud 20.0.5

Updated from an older Nextcloud/ownCloud or fresh install: Started at Nextcloud 15 and updated over the "years" to 20.0.5

Where did you install Nextcloud from: From download page and via updater

Signing status: No errors have been found.

Signing status ``` Login as admin user into your Nextcloud and access http://example.com/index.php/settings/integrity/failed paste the results here. ```

List of activated apps:

App list ``` Enabled: - accessibility: 1.6.0 - activity: 2.13.4 - cloud_federation_api: 1.3.0 - comments: 1.10.0 - contactsinteraction: 1.1.0 - dav: 1.16.2 - federatedfilesharing: 1.10.2 - federation: 1.10.1 - files: 1.15.0 - files_external: 1.11.1 - files_pdfviewer: 2.0.1 - files_rightclick: 0.17.0 - files_sharing: 1.12.2 - files_trashbin: 1.10.1 - files_versions: 1.13.0 - files_videoplayer: 1.9.0 - firstrunwizard: 2.9.0 - logreader: 2.5.0 - lookup_server_connector: 1.8.0 - mail: 1.7.2 - nextcloud_announcements: 1.9.0 - notifications: 2.8.0 - oauth2: 1.8.0 - password_policy: 1.10.1 - photos: 1.2.3 - privacy: 1.4.0 - provisioning_api: 1.10.0 - recommendations: 0.8.0 - serverinfo: 1.10.0 - settings: 1.2.0 - sharebymail: 1.10.0 - support: 1.3.0 - survey_client: 1.8.0 - systemtags: 1.10.0 - text: 3.1.0 - theming: 1.11.0 - twofactor_backupcodes: 1.9.0 - updatenotification: 1.10.0 - user_status: 1.0.1 - viewer: 1.4.0 - weather_status: 1.0.0 - workflowengine: 2.2.0 Disabled: - admin_audit - calendar - dashboard - encryption - user_ldap ```

Nextcloud configuration:

Config report ``` { "system": { "instanceid": "***REMOVED SENSITIVE VALUE***", "passwordsalt": "***REMOVED SENSITIVE VALUE***", "secret": "***REMOVED SENSITIVE VALUE***", "trusted_domains": [ "cloud.xxx.de" ], "datadirectory": "***REMOVED SENSITIVE VALUE***", "dbtype": "mysql", "version": "20.0.5.2", "overwrite.cli.url": "https:\/\/cloud.xxx.de", "dbname": "***REMOVED SENSITIVE VALUE***", "dbhost": "***REMOVED SENSITIVE VALUE***", "dbport": "", "dbtableprefix": "oc_", "dbuser": "***REMOVED SENSITIVE VALUE***", "dbpassword": "***REMOVED SENSITIVE VALUE***", "installed": true, "mail_smtpmode": "smtp", "mail_sendmailmode": "smtp", "mail_from_address": "***REMOVED SENSITIVE VALUE***", "mail_domain": "***REMOVED SENSITIVE VALUE***", "mail_smtphost": "***REMOVED SENSITIVE VALUE***", "mail_smtpport": "25", "maintenance": false, "updater.secret": "***REMOVED SENSITIVE VALUE***", "theme": "", "loglevel": 0, "mysql.utf8mb4": true } } ```

Are you using external storage, if yes which one: local/smb/sftp/... webdav on IONOS hidrive

Are you using encryption: yes/no no

Are you using an external user-backend, if yes which one: LDAP/ActiveDirectory/Webdav/... no

LDAP configuration (delete this part if not used)

LDAP config ``` With access to your command line run e.g.: sudo -u www-data php occ ldap:show-config from within your Nextcloud installation folder Without access to your command line download the data/owncloud.db to your local computer or access your SQL server remotely and run the select query: SELECT * FROM `oc_appconfig` WHERE `appid` = 'user_ldap'; Eventually replace sensitive data as the name/IP-address of your LDAP server or groups. ```

Client configuration

Browser: Chrome Operating system:

Logs

Web server error log

Web server error log ``` [Thu Feb 11 00:00:13.626219 2021] [core:notice] [pid 854] AH00094: Command line: '/usr/sbin/apache2' [Thu Feb 11 00:31:30.328489 2021] [php7:error] [pid 22689] [client 45.155.205.108:47368] script '/var/www/html/index.php' not found or unable to stat [Thu Feb 11 05:50:28.363883 2021] [negotiation:error] [pid 22803] [client 45.155.205.108:34962] AH00687: Negotiation: discovered file(s) matching request: /var/www/html/cloud/console (None could be negotiated). [Thu Feb 11 08:10:12.533052 2021] [php7:error] [pid 22990] [client 123.206.205.78:37604] script '/var/www/html/elrekt.php' not found or unable to stat [Thu Feb 11 08:10:13.010054 2021] [php7:error] [pid 22689] [client 123.206.205.78:37850] script '/var/www/html/index.php' not found or unable to stat [Thu Feb 11 17:39:31.457095 2021] [php7:error] [pid 22803] [client 45.155.205.108:54848] script '/var/www/html/index.php' not found or unable to stat [Thu Feb 11 22:06:56.411540 2021] [access_compat:error] [pid 30262] [client 91.2.56.108:50074] AH01797: client denied by server configuration: /var/www/html/cloud/data/.ocdata [Thu Feb 11 22:07:44.039025 2021] [access_compat:error] [pid 28508] [client 91.2.56.108:50092] AH01797: client denied by server configuration: /var/www/html/cloud/data/.ocdata ```

Nextcloud log (data/nextcloud.log)

Nextcloud log ``` I removed the log file shortly before starting the scan: {"reqId":"4z8SGd5wgB0DyhcDdEb5","level":0,"time":"2021-02-11T21:29:37+00:00","remoteAddr":"91.2.56.xxx","user":"user123","app":"files_external","method":"GET","url":"/ocs/v2.php/apps/notifications/api/v2/notifications","message":"/appinfo/app.php is deprecated, use \\OCP\\AppFramework\\Bootstrap\\IBootstrap on the application class instead.","userAgent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/88.0.4324.146 Safari/537.36","version":"20.0.5.2"} {"reqId":"4z8SGd5wgB0DyhcDdEb5","level":0,"time":"2021-02-11T21:29:37+00:00","remoteAddr":"91.2.56.xxx","user":"user123","app":"files_sharing","method":"GET","url":"/ocs/v2.php/apps/notifications/api/v2/notifications","message":"/appinfo/app.php is deprecated, use \\OCP\\AppFramework\\Bootstrap\\IBootstrap on the application class instead.","userAgent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/88.0.4324.146 Safari/537.36","version":"20.0.5.2"} {"reqId":"afOJ5fVCJbtx4ieCt8WI","level":0,"time":"2021-02-11T21:29:43+00:00","remoteAddr":"","user":"--","app":"files_external","method":"","url":"--","message":"/appinfo/app.php is deprecated, use \\OCP\\AppFramework\\Bootstrap\\IBootstrap on the application class instead.","userAgent":"--","version":"20.0.5.2"} {"reqId":"afOJ5fVCJbtx4ieCt8WI","level":0,"time":"2021-02-11T21:29:43+00:00","remoteAddr":"","user":"--","app":"files_sharing","method":"","url":"--","message":"/appinfo/app.php is deprecated, use \\OCP\\AppFramework\\Bootstrap\\IBootstrap on the application class instead.","userAgent":"--","version":"20.0.5.2"} {"reqId":"uXoRZwLdGi0XXatmfwTn","level":0,"time":"2021-02-11T21:30:07+00:00","remoteAddr":"91.2.56.xxx","user":"user123","app":"files_external","method":"GET","url":"/ocs/v2.php/apps/notifications/api/v2/notifications","message":"/appinfo/app.php is deprecated, use \\OCP\\AppFramework\\Bootstrap\\IBootstrap on the application class instead.","userAgent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/88.0.4324.146 Safari/537.36","version":"20.0.5.2"} {"reqId":"uXoRZwLdGi0XXatmfwTn","level":0,"time":"2021-02-11T21:30:07+00:00","remoteAddr":"91.2.56.xxx","user":"user123","app":"files_sharing","method":"GET","url":"/ocs/v2.php/apps/notifications/api/v2/notifications","message":"/appinfo/app.php is deprecated, use \\OCP\\AppFramework\\Bootstrap\\IBootstrap on the application class instead.","userAgent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/88.0.4324.146 Safari/537.36","version":"20.0.5.2"} {"reqId":"qSFDBAEzeNGOipvuuzkT","level":0,"time":"2021-02-11T21:30:37+00:00","remoteAddr":"91.2.56.xxx","user":"user123","app":"files_external","method":"GET","url":"/ocs/v2.php/apps/notifications/api/v2/notifications","message":"/appinfo/app.php is deprecated, use \\OCP\\AppFramework\\Bootstrap\\IBootstrap on the application class instead.","userAgent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/88.0.4324.146 Safari/537.36","version":"20.0.5.2"} {"reqId":"qSFDBAEzeNGOipvuuzkT","level":0,"time":"2021-02-11T21:30:37+00:00","remoteAddr":"91.2.56.xxx","user":"user123","app":"files_sharing","method":"GET","url":"/ocs/v2.php/apps/notifications/api/v2/notifications","message":"/appinfo/app.php is deprecated, use \\OCP\\AppFramework\\Bootstrap\\IBootstrap on the application class instead.","userAgent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/88.0.4324.146 Safari/537.36","version":"20.0.5.2"} ``` Please let me know if I should produce some other debug information!
szaimen commented 3 years ago

Is this Issue still valid in NC21.0.2? If not, please close this issue. Thanks! :)

ghost commented 3 years ago

This issue has been automatically marked as stale because it has not had recent activity and seems to be missing some essential information. It will be closed if no further activity occurs. Thank you for your contributions.