nextcloud / server

☁️ Nextcloud server, a safe home for all your data
https://nextcloud.com
GNU Affero General Public License v3.0
26.63k stars 3.99k forks source link

occ files:scan --all does not complete, unhandled exception #22787

Closed stanelie closed 1 year ago

stanelie commented 3 years ago

Steps to reproduce

sudo -u www-data php occ files:scan --all

Expected behaviour

File scan should complete without errors

Actual behaviour

unhandled exception is thrown

Server configuration detail

Operating system: Linux 4.4.0-189-generic #219-Ubuntu SMP Tue Aug 11 12:26:50 UTC 2020 x86_64

Webserver: Apache (apache2handler)

Database: mysql 5.7.31

PHP version:

7.2.33-1+ubuntu16.04.1+deb.sury.org+1 Modules loaded: Core, date, libxml, openssl, pcre, zlib, filter, hash, Reflection, SPL, sodium, session, standard, apache2handler, mysqlnd, PDO, xml, apcu, bcmath, bz2, calendar, ctype, curl, dom, mbstring, fileinfo, ftp, gd, gettext, gmp, iconv, igbinary, imagick, imap, intl, json, ldap, exif, memcache, mysqli, pdo_mysql, pdo_sqlite, apc, posix, pspell, readline, recode, redis, shmop, SimpleXML, smbclient, sockets, sqlite3, sysvmsg, sysvsem, sysvshm, tidy, tokenizer, wddx, xmlreader, xmlrpc, xmlwriter, xsl, zip, Phar, libsmbclient, Zend OPcache

Nextcloud version: 19.0.2 - 19.0.2.2

Updated from an older Nextcloud/ownCloud or fresh install: updated

Where did you install Nextcloud from: source download

Signing status Array ( )
List of activated apps ``` Enabled: - audioplayer: 2.11.2 - bruteforcesettings: 2.0.1 - calendar: 2.0.4 - cloud_federation_api: 1.2.0 - comments: 1.9.0 - contacts: 3.3.0 - contactsinteraction: 1.0.0 - dav: 1.15.0 - deck: 1.0.5 - discoursesso: 0.9.22 - federatedfilesharing: 1.9.0 - federation: 1.9.0 - files: 1.14.0 - files_external: 1.10.0 - files_fulltextsearch: 1.4.3 - files_pdfviewer: 1.8.0 - files_retention: 1.8.2 - files_rightclick: 0.16.0 - files_sharing: 1.11.0 - files_trashbin: 1.9.0 - files_versions: 1.12.0 - files_videoplayer: 1.8.0 - firstrunwizard: 2.8.0 - fulltextsearch: 1.4.2 - impersonate: 1.6.1 - issuetemplate: 0.7.0 - logreader: 2.4.0 - lookup_server_connector: 1.7.0 - music: 0.16.0 - nextcloud_announcements: 1.8.0 - notes: 3.6.4 - notifications: 2.7.0 - oauth2: 1.7.0 - password_policy: 1.9.1 - photos: 1.1.0 - privacy: 1.3.0 - provisioning_api: 1.9.0 - quota_warning: 1.8.0 - richdocuments: 3.7.4 - serverinfo: 1.9.0 - settings: 1.1.0 - sharebymail: 1.9.0 - support: 1.2.1 - survey_client: 1.7.0 - systemtags: 1.9.0 - tasks: 0.13.3 - text: 3.0.1 - theming: 1.10.0 - twofactor_backupcodes: 1.8.0 - updatenotification: 1.9.0 - user_ldap: 1.9.0 - viewer: 1.3.0 - workflowengine: 2.1.0 Disabled: - accessibility - activity - admin_audit - apporder - encryption - files_automatedtagging - files_downloadactivity - files_reader - groupfolders - mail - radio - recommendations - social - socialsharing_diaspora - socialsharing_email - socialsharing_facebook - socialsharing_twitter ```
Configuration (config/config.php) ``` { "instanceid": "***REMOVED SENSITIVE VALUE***", "passwordsalt": "***REMOVED SENSITIVE VALUE***", "secret": "***REMOVED SENSITIVE VALUE***", "trusted_domains": [ "10.8.0.5", ], "datadirectory": "***REMOVED SENSITIVE VALUE***", "overwrite.cli.url": "***REMOVED SENSITIVE VALUE***", "dbtype": "mysql", "version": "19.0.2.2", "dbname": "***REMOVED SENSITIVE VALUE***", "dbhost": "***REMOVED SENSITIVE VALUE***", "dbport": "", "dbtableprefix": "oc_", "dbuser": "***REMOVED SENSITIVE VALUE***", "dbpassword": "***REMOVED SENSITIVE VALUE***", "logtimezone": "America\/New_York", "installed": true, "overwriteprotocol": "https", "mail_smtpmode": "smtp", "mail_smtphost": "***REMOVED SENSITIVE VALUE***", "mail_from_address": "***REMOVED SENSITIVE VALUE***", "mail_domain": "***REMOVED SENSITIVE VALUE***", "theme": "", "trashbin_retention_obligation": "7,14", "memcache.local": "\\OC\\Memcache\\Redis", "filelocking.enabled": true, "memcache.locking": "\\OC\\Memcache\\Redis", "redis": { "host": "***REMOVED SENSITIVE VALUE***", "port": 0, "timeout": 0 }, "ldapIgnoreNamingRules": false, "default_language": "fr", "ldapProviderFactory": "\\OCA\\User_LDAP\\LDAPProviderFactory", "trusted_proxies": "***REMOVED SENSITIVE VALUE***", "mail_smtpport": "465", "loglevel": 0, "updater.release.channel": "stable", "maintenance": false, "mail_smtpauthtype": "LOGIN", "mail_smtpsecure": "ssl", "mail_smtpauth": 1, "mail_smtpname": "***REMOVED SENSITIVE VALUE***", "mail_smtppassword": "***REMOVED SENSITIVE VALUE***", "mysql.utf8mb4": true, "app_install_overwrite": [ "calendar", "spreed", "radio", "files_retention", "files_downloadactivity", "discoursesso" ] } ```

Are you using external storage, if yes which one: local, SMB

Are you using encryption:

Are you using an external user-backend, if yes which one: LDAP/ActiveDirectory/Webdav/...

LDAP configuration (delete this par if not used) ``` background_sync_interval: 43200background_sync_offset: 0background_sync_prefix: s01cleanUpJobOffset: 0enabled: yesinstalled_version: 1.9.0s01_lastChange: 1598994901s01has_memberof_filter_support: 1s01home_folder_naming_rule: attr:sAMAccountNames01last_jpegPhoto_lookup: 0s01ldap_agent_password: cXdlOTg3cHBws01ldap_attributes_for_group_search: s01ldap_attributes_for_user_search: s01ldap_backup_host: s01ldap_backup_port: s01ldap_base: DC=casernes01ldap_base_groups: DC=casernes01ldap_base_users: DC=casernes01ldap_cache_ttl: 600s01ldap_configuration_active: 1s01ldap_display_name: displaynames01ldap_dn: CN=owncloud,OU=equipmentusers,OU=SBSUsers,OU=Users,OU=MyBusiness,DC=casernes01ldap_dynamic_group_member_url: s01ldap_email_attr: mails01ldap_experienced_admin: 0s01ldap_expert_username_attr: samaccountnames01ldap_expert_uuid_group_attr: sAMAccountNames01ldap_expert_uuid_user_attr: samaccountnames01ldap_group_display_name: samaccountnames01ldap_group_filter: (&(|(objectclass=group))(|(cn=archives_diamant_rw)(cn=budget_diamant)(cn=inactif_ro)(cn=inactif_rw)(cn=informatique)(cn=ARCHIVES)(cn=owncloud_users)(cn=BUDGETS)(cn=COMMUNICATION)(cn=MBVL)(cn=PRODUCTIONS)(cn=TECHNIQUE)(cn=R&D)(cn=budget_prod)(cn=archives-ro)(cn=COMPTABILITE)))s01ldap_group_filter_mode: 0s01ldap_group_member_assoc_attribute: uniqueMembers01ldap_groupfilter_groups: archives_diamant_rw budget_diamant inactif_ro inactif_rw informatique ARCHIVES owncloud_users BUDGETS COMMUNICATION MBVL PRODUCTIONS TECHNIQUE R&D budget_prod archives-ro COMPTABILITEs01ldap_groupfilter_objectclass: groups01ldap_host: caserne1s01ldap_login_filter: (&(&(|(objectclass=person))(|(|(memberof=CN=owncloud_users,OU=Security Groups,OU=MyBusiness,DC=caserne)(primaryGroupID=5259))))(|(samaccountname=%uid)(|(sAMAccountName=%uid))))s01ldap_login_filter_mode: 0s01ldap_loginfilter_attributes: sAMAccountNames01ldap_loginfilter_email: 0s01ldap_loginfilter_username: 1s01ldap_nested_groups: 0s01ldap_override_main_server: s01ldap_paging_size: 500s01ldap_port: 389s01ldap_quota_attr: s01ldap_quota_def: s01ldap_tls: 0s01ldap_turn_off_cert_check: 0s01ldap_turn_on_pwd_change: 0s01ldap_user_display_name_2: s01ldap_user_filter_mode: 0s01ldap_userfilter_groups: owncloud_userss01ldap_userfilter_objectclass: persons01ldap_userlist_filter: (&(|(objectclass=person))(|(|(memberof=CN=owncloud_users,OU=Security Groups,OU=MyBusiness,DC=caserne)(primaryGroupID=5259))))s01use_memberof_to_detect_membership: 1types: authentication ```

Client configuration

Browser: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/85.0.4183.83 Safari/537.36

Operating system:

Logs

Web server error log ``` Insert your web server log here ```
Nextcloud log ``` An unhandled exception has been thrown: Error: Call to a member function getCache() on null in /var/www/nextcloud/lib/private/Files/Mount/MountPoint.php:272 Stack trace: #0 /var/www/nextcloud/lib/private/Files/Filesystem.php(440): OC\Files\Mount\MountPoint->getStorageRootId() #1 /var/www/nextcloud/lib/private/Files/Filesystem.php(376): OC\Files\Filesystem::initMountPoints('cpatterson') #2 /var/www/nextcloud/lib/private/legacy/OC_Util.php(308): OC\Files\Filesystem::init('cpatterson', '/cpatterson/fil...') #3 /var/www/nextcloud/lib/private/Files/Utils/Scanner.php(116): OC_Util::setupFS('cpatterson') #4 /var/www/nextcloud/lib/private/Files/Utils/Scanner.php(202): OC\Files\Utils\Scanner->getMounts('/cpatterson') #5 /var/www/nextcloud/apps/files/lib/Command/Scan.php(151): OC\Files\Utils\Scanner->scan('/cpatterson', true, NULL) #6 /var/www/nextcloud/apps/files/lib/Command/Scan.php(207): OCA\Files\Command\Scan->scanFiles('cpatterson', '/cpatterson', Object(Symfony\Component\Console\Output\ConsoleOutput), false, true, false) #7 /var/www/nextcloud/3rdparty/symfony/console/Command/Command.php(255): OCA\Files\Command\Scan->execute(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput)) #8 /var/www/nextcloud/core/Command/Base.php(169): Symfony\Component\Console\Command\Command->run(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput)) #9 /var/www/nextcloud/3rdparty/symfony/console/Application.php(1012): OC\Core\Command\Base->run(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput)) #10 /var/www/nextcloud/3rdparty/symfony/console/Application.php(272): Symfony\Component\Console\Application->doRunCommand(Object(OCA\Files\Command\Scan), Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput)) #11 /var/www/nextcloud/3rdparty/symfony/console/Application.php(148): Symfony\Component\Console\Application->doRun(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput)) #12 /var/www/nextcloud/lib/private/Console/Application.php(215): Symfony\Component\Console\Application->run(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput)) #13 /var/www/nextcloud/console.php(100): OC\Console\Application->run() #14 /var/www/nextcloud/occ(11): require_once('/var/www/nextcl...') ```
Browser log Insert your browser log here, this could for example include: a) The javascript console log b) The network log c) ...
yamsu commented 3 years ago

Adding the following line in /var/www/nextcloud/lib/private/Files/Cache/Scanner.php on line 418 solved the issue

if (is_array($fileMeta) == false) continue;

This solution is similar to that of #22497

Not sure if it will cause other errors. But files:scan works without reporting the exception.

With neighboring code

4                 $exceptionOccurred = false;                                    
3                 $childQueue = [];                                              
2                 $newChildNames = [];
1                 foreach ($newChildren as $fileMeta) {                        
418                         if (is_array($fileMeta) == false) continue;
1                         $file = $fileMeta['name'];                             
2                         $newChildNames[] = $file;                              
3                         $child = $path ? $path . '/' . $file : $file;
skjnldsv commented 3 years ago

@yamsu can you open a pull request? :)

szaimen commented 1 year ago

Hi, please update to 24.0.9 or better 25.0.3 and report back if it fixes the issue. Thank you!

My goal is to add a label like e.g. 25-feedback to this ticket of an up-to-date major Nextcloud version where the bug could be reproduced. However this is not going to work without your help. So thanks for all your effort!

If you don't manage to reproduce the issue in time and the issue gets closed but you can reproduce the issue afterwards, feel free to create a new bug report with up-to-date information by following this link: https://github.com/nextcloud/server/issues/new?assignees=&labels=bug%2C0.+Needs+triage&template=BUG_REPORT.yml&title=%5BBug%5D%3A+