nextcloud / server

☁️ Nextcloud server, a safe home for all your data
https://nextcloud.com
GNU Affero General Public License v3.0
27.39k stars 4.07k forks source link

[Bug]: Nextcloud 28: Files instable appear / disappear #43004

Closed nicovd737 closed 9 months ago

nicovd737 commented 9 months ago

⚠️ This issue respects the following points: ⚠️

Bug description

When accessing Files module in Nextcloud, some users can't see files at all whereas they are available in storage. When refreshing the page, sometime they appears again but disappears again after some time. It happens for all files (locally stored on nextcloud or external file or shared files) Moreover, it's seems that the web page freeze because when we click to shares or external storage or even into a directory, nothing move. To be honnest, i don't know if it's a pure nextcloud issue, or nginx or redis, ... Note: This is not happening for all users. Some users don't have any issue. Thanks for your help.

nextcloud_files1

Error in webbrowser console:

nextcloud_files4

Steps to reproduce

  1. Upgrade from Nextcloud 27 to Nextcloud 28 (docker image)
  2. Go to Files
  3. See if you have your files on the screen.

Expected behavior

Files should be presented to users without any issue.

Installation method

Community Docker image

Nextcloud Server version

28

Operating system

Debian/Ubuntu

PHP engine version

PHP 8.2

Web server

Nginx

Database engine version

MariaDB

Is this bug present after an update or on a fresh install?

Upgraded to a MAJOR version (ex. 22 to 23)

Are you using the Nextcloud Server Encryption module?

None

What user-backends are you using?

Configuration report

{
    "system": {
        "instanceid": "***REMOVED SENSITIVE VALUE***",
        "passwordsalt": "***REMOVED SENSITIVE VALUE***",
        "secret": "***REMOVED SENSITIVE VALUE***",
        "trusted_domains": [
            "cloud.payerne.swiss"
        ],
        "datadirectory": "***REMOVED SENSITIVE VALUE***",
        "dbtype": "mysql",
        "version": "28.0.1.1",
        "overwriteprotocol": "https",
        "dbname": "***REMOVED SENSITIVE VALUE***",
        "dbhost": "***REMOVED SENSITIVE VALUE***",
        "dbport": "3306",
        "dbtableprefix": "oc_",
        "dbuser": "***REMOVED SENSITIVE VALUE***",
        "dbpassword": "***REMOVED SENSITIVE VALUE***",
        "installed": true,
        "ldapIgnoreNamingRules": false,
        "ldapProviderFactory": "\\OCA\\User_LDAP\\LDAPProviderFactory",
        "mail_smtpmode": "smtp",
        "mail_smtpauthtype": "NTLM",
        "mail_from_address": "***REMOVED SENSITIVE VALUE***",
        "mail_domain": "***REMOVED SENSITIVE VALUE***",
        "mail_smtphost": "***REMOVED SENSITIVE VALUE***",
        "mail_smtpport": "10587",
        "default_phone_region": "CH",
        "filelocking.enabled": true,
        "memcache.local": "\\OC\\Memcache\\APCu",
        "memcache.locking": "\\OC\\Memcache\\Redis",
        "redis": {
            "host": "***REMOVED SENSITIVE VALUE***",
            "password": "***REMOVED SENSITIVE VALUE***",
            "port": 6379
        },
        "trusted_proxies": "***REMOVED SENSITIVE VALUE***",
        "overwrite.cli.url": "https:\/\/cloud.payerne.swiss",
        "htaccess.RewriteBase": "\/",
        "maintenance": false,
        "theme": "",
        "loglevel": 0,
        "mail_smtpauth": 1,
        "mail_smtpname": "***REMOVED SENSITIVE VALUE***",
        "mail_smtppassword": "***REMOVED SENSITIVE VALUE***",
        "updater.release.channel": "stable",
        "twofactor_enforced": "false",
        "twofactor_enforced_groups": [
            "admin",
            "SDIS"
        ],
        "twofactor_enforced_excluded_groups": [],
        "mysql.utf8mb4": true,
        "app_install_overwrite": [
            "spreed",
            "twofactor_rcdevsopenotp",
            "occweb",
            "mindmap_app",
            "drawio",
            "officeonline",
            "files_mindmap",
            "openotp_auth",
            "impersonate",
            "integration_moodle",
            "talk_matterbridge",
            "files_downloadactivity",
            "imageconverter",
            "emlviewer"
        ],
        "apps_paths": [
            {
                "path": "\/var\/www\/html\/apps",
                "url": "\/apps",
                "writable": false
            },
            {
                "path": "\/var\/www\/html\/custom_apps",
                "url": "\/custom_apps",
                "writable": true
            }
        ],
        "memcache.distributed": "\\OC\\Memcache\\Redis",
        "data-fingerprint": "550994713cc295254572150adaf2b32c",
        "logtimezone": "CEST",
        "mail_sendmailmode": "smtp",
        "memories.exiftool": "\/var\/www\/html\/custom_apps\/memories\/bin-ext\/exiftool-amd64-glibc",
        "memories.vod.path": "\/var\/www\/html\/custom_apps\/memories\/bin-ext\/go-vod-amd64",
        "simpleSignUpLink.shown": false
    }
}

List of activated Apps

Enabled:
  - activity: 2.20.0
  - admin_audit: 1.18.0
  - announcementcenter: 6.7.0
  - audioplayer: 3.4.1
  - bruteforcesettings: 2.8.0
  - calendar: 4.6.4
  - circles: 28.0.0-dev
  - cloud_federation_api: 1.11.0
  - comments: 1.18.0
  - contacts: 5.5.1
  - contactsinteraction: 1.9.0
  - dashboard: 7.8.0
  - dav: 1.29.1
  - deck: 1.12.1
  - drawio: 3.0.2
  - external: 5.3.1
  - federatedfilesharing: 1.18.0
  - federation: 1.18.0
  - files: 2.0.0
  - files_accesscontrol: 1.18.0
  - files_automatedtagging: 1.18.0
  - files_external: 1.20.0
  - files_fulltextsearch: 28.0.0
  - files_mindmap: 0.0.30
  - files_pdfviewer: 2.9.0
  - files_reminders: 1.1.0
  - files_retention: 1.17.0
  - files_sharing: 1.20.0
  - files_trashbin: 1.18.0
  - files_versions: 1.21.0
  - firstrunwizard: 2.17.0
  - forms: 4.0.0
  - geoblocker: 0.5.13
  - groupfolders: 16.0.3
  - imageconverter: 1.3.5
  - impersonate: 1.15.0
  - integration_moodle: 1.0.2
  - logreader: 2.13.0
  - lookup_server_connector: 1.16.0
  - mail: 3.5.4
  - memories: 6.2.2
  - nextcloud_announcements: 1.17.0
  - notes: 4.9.2
  - notifications: 2.16.0
  - oauth2: 1.16.3
  - occweb: 0.1.1
  - officeonline: 2.0.3
  - openotp_auth: 1.5.1
  - passman: 2.4.8
  - password_policy: 1.18.0
  - photos: 2.4.0
  - polls: 6.0.1
  - privacy: 1.12.0
  - provisioning_api: 1.18.0
  - recommendations: 2.0.0
  - related_resources: 1.3.0
  - serverinfo: 1.18.0
  - settings: 1.10.1
  - sharebymail: 1.18.0
  - side_menu: 3.11.3
  - smb_test: 0.3.8
  - spreed: 18.0.1
  - survey_client: 1.16.0
  - systemtags: 1.18.0
  - talk_matterbridge: 1.26.0
  - tasks: 0.15.0
  - text: 3.9.1
  - theming: 2.3.0
  - theming_customcss: 1.15.0
  - twofactor_backupcodes: 1.17.0
  - twofactor_nextcloud_notification: 3.8.0
  - updatenotification: 1.18.0
  - user_ldap: 1.19.0
  - user_status: 1.8.1
  - viewer: 2.2.0
  - weather_status: 1.8.0
  - workflowengine: 2.10.0
Disabled:
  - emlviewer: 1.0.8 (installed 1.0.8)
  - encryption: 2.16.0 (installed 2.10.0)
  - extract: 1.3.6 (installed 1.3.6)
  - files_downloadactivity: 1.16.0 (installed 1.16.0)
  - files_rightclick: 0.15.1 (installed 1.6.0)
  - fulltextsearch: 28.0.0 (installed 28.0.0)
  - fulltextsearch_elasticsearch: 28.0.0 (installed 28.0.0)
  - ransomware_protection: 1.14.0 (installed 1.14.0)
  - support: 1.11.0 (installed 1.5.0)
  - suspicious_login: 6.0.0
  - twofactor_totp: 10.0.0-beta.2 (installed 6.4.0)

Nextcloud Signing status

Technical information
=====================
The following list covers which files have failed the integrity check. Please read
the previous linked documentation to learn more about the errors and how to fix
them.

Results
=======
- openotp
    - EXCEPTION
        - Exception
        - App not found
- OpenOTP Two Factor Authentication
    - EXCEPTION
        - Exception
        - App not found

Raw output
==========
Array
(
    [openotp] => Array
        (
            [EXCEPTION] => Array
                (
                    [class] => Exception
                    [message] => App not found
                )

        )

    [OpenOTP Two Factor Authentication] => Array
        (
            [EXCEPTION] => Array
                (
                    [class] => Exception
                    [message] => App not found
                )

        )

)

Nextcloud Logs

Any solution to anonymise data on this files ? There too much critical info in it. I don't want this to be public.

Additional info

Happens on all browsers and OS.

solracsf commented 9 months ago

Any solution to anonymise data on this files ? There too much critical info in it. I don't want this to be public.

Some already are; otherwise, by hand with a search & replace editor. But logs are crucial to help...

nicovd737 commented 9 months ago

nextcloud.log

nicovd737 commented 9 months ago

Hello,

Found something. In the Files app settings, there was a check box activated for "Display folder description". When i unchek this one, all files appears again and works correctly.

Can this be a bug ?

joshtrichards commented 9 months ago

Found something. In the Files app settings, there was a check box activated for "Display folder description".

I can't reproduce this.

But you may want to start my disabling some of the apps you have force enabled that aren't compatible with NC28. These are all force enabled in your config:

        "app_install_overwrite": [
            "spreed",
            "twofactor_rcdevsopenotp",
            "occweb",
            "mindmap_app",
            "drawio",
            "officeonline",
            "files_mindmap",
            "openotp_auth",
            "impersonate",
            "integration_moodle",
            "talk_matterbridge",
            "files_downloadactivity",
            "imageconverter",
            "emlviewer"
        ],

I see at least one of these actively failing in your logs (openotp_auth) and off the to of my head I see a couple on there that I also don't believe are compatible.

Please re-test after you've confirmed all activated apps are compatible.

nicovd737 commented 9 months ago

Found something. In the Files app settings, there was a check box activated for "Display folder description".

I can't reproduce this.

But you may want to start my disabling some of the apps you have force enabled that aren't compatible with NC28. These are all force enabled in your config:

        "app_install_overwrite": [
            "spreed",
            "twofactor_rcdevsopenotp",
            "occweb",
            "mindmap_app",
            "drawio",
            "officeonline",
            "files_mindmap",
            "openotp_auth",
            "impersonate",
            "integration_moodle",
            "talk_matterbridge",
            "files_downloadactivity",
            "imageconverter",
            "emlviewer"
        ],

I see at least one of these actively failing in your logs (openotp_auth) and off the to of my head I see a couple on there that I also don't believe are compatible.

Please re-test after you've confirmed all activated apps are compatible.

Hello, Thanks for your feedback. Yes i know that lot of apps are not compatible with NC28 wich is ... disappointing. Dev have to think that there are lot of users in nextcloud (not IT guys). We can't say to our user "Oh sorry, officeline stop, drawio stop, 2FA stop, ..." That's not the good way to work, that's not a good pub for NC i would say. You can close that subject as for me, it's working ok with files now. Regards, Nico

joshtrichards commented 9 months ago

Just to make sure I didn't create some additional confusion, I wasn't implying all those listed apps are incompatible today. Being listed there just means they were force enabled at some point in the past - and that compatibility checks no longer get performed if they're listed there during Server upgrades.

It looks like drawio is available for NC28, but not sure about the others. Looks like 2FA you're using is something from @rcdevs that integrates with OpenOTP - do you have an enterprise arrangement with them? If so, probably just reach out to them.

I hear you though. It can be a challenge navigating the differences of various components. Fortunately, 2-3 major versions of Nextcloud Server are always well supported at a given point in time (v26/v27/v28 at the time). Usually no need to go to the latest and greatest until one's favorite apps are ready for it as well. :-)

Glad to hear Nextcloud Server is working again for you!

One idea to prevent the failure state you ended up in from occurring is to clear all force enabled apps (i.e. clear that config parameter) such as proposed in #43026. That way force enabled apps aren't carried through forever, leading to problems like you experienced within Server itself. That way major version upgrades can't break from incompatible apps at least.

rcdevs commented 9 months ago

Hello everybody,

Unfortunately, our MFA plugin for Nextcloud is not yet compatible with version 28. We are actively working on providing an updated version that works seamlessly with Nextcloud 28. Dealing with NC updates can be a bit challenging; each time a new NC version is released, we have to address issues and release a new version of the plugin, even if nothing has changed on our end. It would be beneficial if Nextcloud could support retro-compatibility for plugin providers. Apologies for any inconvenience caused.

Regards