[Bug]: Not possible to delete a remnant if no active LDAP configuration are present

kernstock commented 5 months ago

⚠️ This issue respects the following points: ⚠️

[X] This is a bug, not a question or a configuration/webserver/proxy issue.
[X] This issue is not already reported on Github OR Nextcloud Community Forum (I've searched it).
[X] Nextcloud Server is up to date. See Maintenance and Release Schedule for supported versions.
[X] I agree to follow Nextcloud's Code of Conduct.

Bug description

The directory server of our organization has let's say disappeared. We continued to use the Nextcloud instance but with the built-in user backend.

We now still see account remnants of the disappeared directory in terms of rows in the oc_accounts table as well as user directories in the data directory.

We also see entries in the log where it says "User 042E7FA3-6B7D-45BD-A764-197079C316C9 still has unscanned files after running background scan, background scan might be stopped prematurely"

I tried occ user:sync-account-data but without success. Also occ user:list does not show the remnants.

Is there a safe, manual way to get rid of these remnants?

Steps to reproduce

Set up Nextcloud with a functioning LDAP user backend connected to a directory and user from there.
Kill down the LDAP directory.
Disable the LDAP user backend.
Create new users in the default database backend.

Expected behavior

Users/accounts from the directory should be purged. At least occ user:sync-account-data command should notice and purge the accounts.

Installation method

Community Manual installation with Archive

Nextcloud Server version

27

Operating system

Debian/Ubuntu

PHP engine version

PHP 8.1

Web server

Nginx

Database engine version

MariaDB

Is this bug present after an update or on a fresh install?

None

Are you using the Nextcloud Server Encryption module?

Encryption is Disabled

What user-backends are you using?

[X] Default user-backend (database)
[ ] LDAP/ Active Directory
[ ] SSO - SAML
[ ] Other

Configuration report

{
    "system": {
        "debug": false,
        "loglevel": 2,
        "log_rotate_size": 104857600,
        "instanceid": "***REMOVED SENSITIVE VALUE***",
        "passwordsalt": "***REMOVED SENSITIVE VALUE***",
        "secret": "***REMOVED SENSITIVE VALUE***",
        "trusted_domains": [
            "***REMOVED SENSITIVE VALUE***",
        ],
        "datadirectory": "***REMOVED SENSITIVE VALUE***",
        "dbtype": "mysql",
        "version": "27.1.7.2",
        "overwriteprotocol": "https",
        "dbname": "***REMOVED SENSITIVE VALUE***",
        "dbhost": "***REMOVED SENSITIVE VALUE***",
        "dbport": "",
        "dbtableprefix": "oc_",
        "mysql.utf8mb4": true,
        "dbuser": "***REMOVED SENSITIVE VALUE***",
        "dbpassword": "***REMOVED SENSITIVE VALUE***",
        "installed": true,
        "overwrite.cli.url": "***REMOVED SENSITIVE VALUE***",,
        "ldapIgnoreNamingRules": false,
        "mail_smtpmode": "smtp",
        "mail_sendmailmode": "smtp",
        "maintenance": false,
        "filelocking.enabled": true,
        "memcache.local": "\\OC\\Memcache\\APCu",
        "memcache.locking": "\\OC\\Memcache\\Redis",
        "memcache.distributed": "\\OC\\Memcache\\Redis",
        "redis": {
            "host": "***REMOVED SENSITIVE VALUE***",
            "port": 0,
            "dbindex": 0,
            "timeout": 1.5
        },
        "theme": "",
        "default_phone_region": "GB",
        "app_install_overwrite": [
            "spreed",
            "calendar",
            "jitsi"
        ],
        "updater.release.channel": "stable",
        "preview_max_memory": 512,
        "mail_from_address": "***REMOVED SENSITIVE VALUE***",
        "mail_domain": "***REMOVED SENSITIVE VALUE***",
        "mail_smtpauthtype": "LOGIN",
        "mail_smtpauth": 1,
        "mail_smtphost": "***REMOVED SENSITIVE VALUE***",
        "mail_smtpport": "587",
        "mail_smtpname": "***REMOVED SENSITIVE VALUE***",
        "mail_smtppassword": "***REMOVED SENSITIVE VALUE***",
        "mail_smtpsecure": "tls"
    }
}

List of activated Apps

Enabled:
  - activity: 2.19.0
  - announcementcenter: 6.8.0
  - bruteforcesettings: 2.7.0
  - calendar: 4.6.6
  - calendar_resource_management: 0.6.0
  - circles: 27.0.1
  - cloud_federation_api: 1.10.0
  - collectives: 2.10.0
  - comments: 1.17.0
  - contacts: 5.5.3
  - contactsinteraction: 1.8.0
  - dashboard: 7.7.0
  - dav: 1.27.0
  - deck: 1.11.4
  - federatedfilesharing: 1.17.0
  - federation: 1.17.0
  - files: 1.22.0
  - files_accesscontrol: 1.17.1
  - files_automatedtagging: 1.17.0
  - files_external: 1.19.0
  - files_pdfviewer: 2.8.0
  - files_reminders: 1.0.0
  - files_retention: 1.16.0
  - files_rightclick: 1.6.0
  - files_sharing: 1.19.0
  - files_trashbin: 1.17.0
  - files_versions: 1.20.0
  - firstrunwizard: 2.16.0
  - forms: 3.4.6
  - groupfolders: 15.3.5
  - integration_gitlab: 1.0.19
  - integration_openproject: 2.6.1
  - jitsi: 0.18.0
  - logreader: 2.12.0
  - lookup_server_connector: 1.15.0
  - mail: 3.5.7
  - nextcloud_announcements: 1.16.0
  - notes: 4.9.2
  - notifications: 2.15.0
  - oauth2: 1.15.2
  - onlyoffice: 8.2.4
  - password_policy: 1.17.0
  - passwords: 2023.12.32
  - polls: 5.4.3
  - privacy: 1.11.0
  - provisioning_api: 1.17.0
  - recommendations: 1.6.0
  - related_resources: 1.2.0
  - serverinfo: 1.17.0
  - settings: 1.9.0
  - sharebymail: 1.17.0
  - spreed: 17.1.6
  - support: 1.10.0
  - survey_client: 1.15.0
  - systemtags: 1.17.0
  - tasks: 0.15.0
  - text: 3.8.0
  - theming: 2.2.0
  - twofactor_backupcodes: 1.16.0
  - updatenotification: 1.17.0
  - user_status: 1.7.0
  - viewer: 2.1.0
  - weather_status: 1.7.0
  - workflowengine: 2.9.0
Disabled:
  - admin_audit: 1.17.0
  - encryption: 2.15.0
  - photos: 2.3.0 (installed 1.2.3)
  - suspicious_login: 5.0.0
  - twofactor_totp: 9.0.0
  - user_ldap: 1.17.0 (installed 1.14.1)

Nextcloud Signing status

No errors have been found.

Nextcloud Logs

No response

Additional info

No response

kesselb commented 5 months ago

Did you try https://docs.nextcloud.com/server/latest/admin_manual/configuration_user/user_auth_ldap_cleanup.html?

kernstock commented 5 months ago

Did you try https://docs.nextcloud.com/server/latest/admin_manual/configuration_user/user_auth_ldap_cleanup.html?

You mean reactivate the LDAP user backend and set ldapUserCleanupInterval? Actually, I am hesitant doing this because the LDAP backend has a history of making the entire instance completely inaccessible when a malfunctioning configuration is present. I am unsure whether our old directory config is still present somewhere in the database. When the directory disappeared, I had to disable the LDAP backend with occ only to be able to log in with the admin user (which did not come from the directory server).

kesselb commented 5 months ago

To use occ ldap:show-remnants the user_ldap app must be enabled.

Did you see the part "Deleting local Nextcloud users" that explains how to delete the remnants?

kernstock commented 5 months ago

To use occ ldap:show-remnants the user_ldap app must be enabled.

Yes, I am aware of that (see above).

Did you see the part "Deleting local Nextcloud users" that explains how to delete the remnants?

Yes, trying to delete a remnant this way gives me "User does not exist":

> occ user:delete 042E7FA3-6B7D-45BD-A764-197079C316C9
User does not exist

kesselb commented 5 months ago

To delete a user who belongs to the ldap user backend, the user_ldap app must be enabled.

kesselb commented 5 months ago

There are also a couple of occ commands to manage the ldap configuration, just in case that enabling the user_ldap app makes your instance unusable.

kernstock commented 5 months ago

@kesselb Thanks. In this case, I will have to announce downtime, try reactivating the user_ldap app (which is currently not possible it seems; "The library ldap is not available"). I will report back.

just in case that enabling the user_ldap app makes your instance unusable.

How is this even a thing? In case you accidentally screwed your LDAP config, or - this may happen as we know - the directory disappears. Why is then nobody able to log in to the instance to eventually disable the user_ldap app? In cases where you have Nextcloud provided by a service provider and have no access to the server's command line, this completely locks everyone out out of everything. And leaves instance admins hanging in the providers service queues.

daniel-lerch commented 5 months ago

To delete a user who belongs to the ldap user backend, the user_ldap app must be enabled.

It is not sufficient if user_ldap is enabled. For me the manual delete with occ user:delete UID does not work even when the app is enabled:
User does not exist

I had to configure an LDAP server to delete a user. This does not make sense to me but that is the way it is implemented as of now.

kesselb commented 5 months ago

cc @nextcloud/ldap

come-nc commented 5 months ago

It is expected that the user_ldap application should be enabled in order to delete its users, we cannot do otherwise if we want the deletion to be clean.

https://docs.nextcloud.com/server/latest/admin_manual/configuration_user/user_auth_ldap_cleanup.html is the correct reference here.

The user_ldap application is supporting multiple configuration and I expect this is why when there is no configurations at all it misbehaves and say the user does not exist. You may open an issue for that but I see it as low priority as it’s quite a corner case and can be worked around easily.

kernstock commented 5 months ago

@come-nc Thanks for your answer. I agree that this is quite an edge case. But how can it be worked around easily? Admins that have remnants from a disappeared directory, will they have to spin up an empty directory for the user_ldap app to be able to recognize and delete them? This doesn't seem practicable to me.

come-nc commented 5 months ago

Hum yeah you’re right, I thought an empty configuration would be enough but user_ldap needs an active configuration to behave correctly. We should make user:delete more robust to empty ldap configuration.

come-nc commented 5 months ago

That would mean moving userExists from User_LDAP to User_Proxy directly I’d say, since it does not depend on anything configuration specific, apart from cache currently but caching that globally would be fine. I do not have time to look more into it for now.

nextcloud / server