[Bug]: Unable to enable `user_ldap` app - PostgreSQL error: index "primary" does not exist

skid9000 commented 1 year ago

⚠️ This issue respects the following points: ⚠️

[X] This is a bug, not a question or a configuration/webserver/proxy issue.
[X] This issue is not already reported on Github (I've searched it).
[X] Nextcloud Server is up to date. See Maintenance and Release Schedule for supported versions.
[X] Nextcloud Server is running on 64bit capable CPU, PHP and OS.
[X] I agree to follow Nextcloud's Code of Conduct.

Bug description

I'm unable to enable the user_ldap app, resulting with the following error : An exception occurred while executing a query: SQLSTATE[42704]: Undefined object: 7 ERROR: index "primary" does not exist

Steps to reproduce

./occ app:enable user_ldap
Get the error

Expected behavior

The app should enable.

Installation method

Community Manual installation with Archive

Operating system

Other

PHP engine version

PHP 8.1

Web server

Nginx

Database engine version

PostgreSQL

Is this bug present after an update or on a fresh install?

None

Are you using the Nextcloud Server Encryption module?

Encryption is Disabled

What user-backends are you using?

[ ] Default user-backend (database)
[ ] LDAP/ Active Directory
[ ] SSO - SAML
[ ] Other

Configuration report

{
    "system": {
        "instanceid": "***REMOVED SENSITIVE VALUE***",
        "passwordsalt": "***REMOVED SENSITIVE VALUE***",
        "secret": "***REMOVED SENSITIVE VALUE***",
        "trusted_domains": [
            "***REMOVED SENSITIVE VALUE***"
        ],
        "datadirectory": "***REMOVED SENSITIVE VALUE***",
        "overwrite.cli.url": "***REMOVED SENSITIVE VALUE***",
        "dbtype": "pgsql",
        "version": "25.0.4.1",
        "dbname": "***REMOVED SENSITIVE VALUE***",
        "dbhost": "***REMOVED SENSITIVE VALUE***",
        "dbport": "",
        "dbtableprefix": "oc_",
        "mysql.utf8mb4": true,
        "dbuser": "***REMOVED SENSITIVE VALUE***",
        "installed": true,
        "maintenance": false,
        "theme": "",
        "loglevel": 2,
        "memcache.local": "\\OC\\Memcache\\APCu",
        "memcache.locking": "\\OC\\Memcache\\Redis",
        "redis": {
            "host": "***REMOVED SENSITIVE VALUE***",
            "port": 6379
        },
        "debug": "false",
        "updater.release.channel": "stable",
        "app_install_overwrite": [
            "ocsms",
            "sensorlogger",
            "contacts",
            "unsplash"
        ],
        "encryption.key_storage_migrated": false,
        "default_phone_region": "FR",
        "filesystem_check_changes": 1,
        "mail_smtpmode": "smtp",
        "mail_smtpauthtype": "PLAIN",
        "mail_sendmailmode": "smtp",
        "mail_smtpauth": 1,
        "mail_smtphost": "***REMOVED SENSITIVE VALUE***",
        "mail_smtpport": "465",
        "mail_smtpsecure": "ssl",
        "mail_from_address": "***REMOVED SENSITIVE VALUE***",
        "mail_domain": "***REMOVED SENSITIVE VALUE***",
        "mail_smtpname": "***REMOVED SENSITIVE VALUE***",
        "mail_smtppassword": "***REMOVED SENSITIVE VALUE***",
        "updater.secret": "***REMOVED SENSITIVE VALUE***"
    }
}

List of activated Apps

Enabled:
  - activity: 2.17.0
  - admin_audit: 1.15.0
  - audioplayer: 3.3.1
  - calendar: 4.2.4
  - circles: 25.0.0
  - cloud_federation_api: 1.8.0
  - comments: 1.15.0
  - contacts: 5.1.0
  - contactsinteraction: 1.6.0
  - dashboard: 7.5.0
  - dav: 1.24.0
  - encryption: 2.13.0
  - federatedfilesharing: 1.15.0
  - files: 1.20.1
  - files_external: 1.17.0
  - files_pdfviewer: 2.6.0
  - files_rightclick: 1.4.0
  - files_sharing: 1.17.0
  - files_trashbin: 1.15.0
  - logreader: 2.10.0
  - lookup_server_connector: 1.13.0
  - maps: 0.2.4
  - nextcloud_announcements: 1.14.0
  - notes: 4.6.0
  - notifications: 2.13.1
  - oauth2: 1.13.0
  - ocsms: 2.2.0
  - password_policy: 1.15.0
  - photos: 2.0.1
  - privacy: 1.9.0
  - provisioning_api: 1.15.0
  - recommendations: 1.4.0
  - related_resources: 1.0.4
  - serverinfo: 1.15.0
  - settings: 1.7.0
  - support: 1.8.0
  - systemtags: 1.15.0
  - text: 3.6.0
  - theming: 2.0.1
  - twofactor_backupcodes: 1.14.0
  - twofactor_totp: 7.0.0
  - updatenotification: 1.15.0
  - user_status: 1.5.0
  - viewer: 1.9.0
  - weather_status: 1.5.0
  - workflowengine: 2.7.0
Disabled:
  - bruteforcesettings: 2.3.0
  - federation: 1.3.0
  - files_versions: 1.16.0
  - firstrunwizard: 2.2.1
  - gpxedit: 0.0.14
  - gpxpod: 5.0.4
  - keeweb: 0.6.10
  - sharebymail: 1.3.0
  - survey_client: 1.1.0
  - suspicious_login
  - user_ldap

Nextcloud Signing status

No errors have been found.

Nextcloud Logs

{"reqId":"HnUMISJJgQ7P51ipL9jy","level":3,"time":"2023-02-25T16:19:22+00:00","remoteAddr":"REDACTED","user":"skid","app":"settings","method":"POST","url":"/settings/apps/enable","message":"could not enable apps","userAgent":"Mozilla/5.0 (X11; Linux x86_64; rv:109.0) Gecko/20100101 Firefox/110.0","version":"25.0.4.1","exception":{"Exception":"Doctrine\\DBAL\\Exception\\DriverException","Message":"An exception occurred while executing a query: SQLSTATE[42704]: Undefined object: 7 ERROR:  index \"primary\" does not exist","Code":7,"Trace":[{"file":"/usr/home/nextcloud/live/3rdparty/doctrine/dbal/src/Connection.php","line":1780,"function":"convert","class":"Doctrine\\DBAL\\Driver\\API\\PostgreSQL\\ExceptionConverter","type":"->"},{"file":"/usr/home/nextcloud/live/3rdparty/doctrine/dbal/src/Connection.php","line":1719,"function":"handleDriverException","class":"Doctrine\\DBAL\\Connection","type":"->"},{"file":"/usr/home/nextcloud/live/3rdparty/doctrine/dbal/src/Connection.php","line":1067,"function":"convertExceptionDuringQuery","class":"Doctrine\\DBAL\\Connection","type":"->"},{"file":"/usr/home/nextcloud/live/lib/private/DB/Connection.php","line":261,"function":"executeQuery","class":"Doctrine\\DBAL\\Connection","type":"->"},{"file":"/usr/home/nextcloud/live/3rdparty/doctrine/dbal/src/Connection.php","line":1809,"function":"executeQuery","class":"OC\\DB\\Connection","type":"->"},{"file":"/usr/home/nextcloud/live/lib/private/DB/Migrator.php","line":162,"function":"query","class":"Doctrine\\DBAL\\Connection","type":"->"},{"file":"/usr/home/nextcloud/live/lib/private/DB/Migrator.php","line":69,"function":"applySchema","class":"OC\\DB\\Migrator","type":"->"},{"file":"/usr/home/nextcloud/live/lib/private/DB/Connection.php","line":582,"function":"migrate","class":"OC\\DB\\Migrator","type":"->"},{"file":"/usr/home/nextcloud/live/lib/private/DB/MigrationService.php","line":454,"function":"migrateToSchema","class":"OC\\DB\\Connection","type":"->"},{"file":"/usr/home/nextcloud/live/lib/private/DB/MigrationService.php","line":408,"function":"migrateSchemaOnly","class":"OC\\DB\\MigrationService","type":"->"},{"file":"/usr/home/nextcloud/live/lib/private/Installer.php","line":153,"function":"migrate","class":"OC\\DB\\MigrationService","type":"->"},{"file":"/usr/home/nextcloud/live/apps/settings/lib/Controller/AppSettingsController.php","line":451,"function":"installApp","class":"OC\\Installer","type":"->"},{"file":"/usr/home/nextcloud/live/lib/private/AppFramework/Http/Dispatcher.php","line":225,"function":"enableApps","class":"OCA\\Settings\\Controller\\AppSettingsController","type":"->"},{"file":"/usr/home/nextcloud/live/lib/private/AppFramework/Http/Dispatcher.php","line":133,"function":"executeController","class":"OC\\AppFramework\\Http\\Dispatcher","type":"->"},{"file":"/usr/home/nextcloud/live/lib/private/AppFramework/App.php","line":172,"function":"dispatch","class":"OC\\AppFramework\\Http\\Dispatcher","type":"->"},{"file":"/usr/home/nextcloud/live/lib/private/Route/Router.php","line":298,"function":"main","class":"OC\\AppFramework\\App","type":"::"},{"file":"/usr/home/nextcloud/live/lib/base.php","line":1047,"function":"match","class":"OC\\Route\\Router","type":"->"},{"file":"/usr/home/nextcloud/live/index.php","line":36,"function":"handleRequest","class":"OC","type":"::"}],"File":"/usr/home/nextcloud/live/3rdparty/doctrine/dbal/src/Driver/API/PostgreSQL/ExceptionConverter.php","Line":83,"Previous":{"Exception":"Doctrine\\DBAL\\Driver\\PDO\\Exception","Message":"SQLSTATE[42704]: Undefined object: 7 ERROR:  index \"primary\" does not exist","Code":7,"Trace":[{"file":"/usr/home/nextcloud/live/3rdparty/doctrine/dbal/src/Driver/PDO/Connection.php","line":87,"function":"new","class":"Doctrine\\DBAL\\Driver\\PDO\\Exception","type":"::"},{"file":"/usr/home/nextcloud/live/3rdparty/doctrine/dbal/src/Connection.php","line":1062,"function":"query","class":"Doctrine\\DBAL\\Driver\\PDO\\Connection","type":"->"},{"file":"/usr/home/nextcloud/live/lib/private/DB/Connection.php","line":261,"function":"executeQuery","class":"Doctrine\\DBAL\\Connection","type":"->"},{"file":"/usr/home/nextcloud/live/3rdparty/doctrine/dbal/src/Connection.php","line":1809,"function":"executeQuery","class":"OC\\DB\\Connection","type":"->"},{"file":"/usr/home/nextcloud/live/lib/private/DB/Migrator.php","line":162,"function":"query","class":"Doctrine\\DBAL\\Connection","type":"->"},{"file":"/usr/home/nextcloud/live/lib/private/DB/Migrator.php","line":69,"function":"applySchema","class":"OC\\DB\\Migrator","type":"->"},{"file":"/usr/home/nextcloud/live/lib/private/DB/Connection.php","line":582,"function":"migrate","class":"OC\\DB\\Migrator","type":"->"},{"file":"/usr/home/nextcloud/live/lib/private/DB/MigrationService.php","line":454,"function":"migrateToSchema","class":"OC\\DB\\Connection","type":"->"},{"file":"/usr/home/nextcloud/live/lib/private/DB/MigrationService.php","line":408,"function":"migrateSchemaOnly","class":"OC\\DB\\MigrationService","type":"->"},{"file":"/usr/home/nextcloud/live/lib/private/Installer.php","line":153,"function":"migrate","class":"OC\\DB\\MigrationService","type":"->"},{"file":"/usr/home/nextcloud/live/apps/settings/lib/Controller/AppSettingsController.php","line":451,"function":"installApp","class":"OC\\Installer","type":"->"},{"file":"/usr/home/nextcloud/live/lib/private/AppFramework/Http/Dispatcher.php","line":225,"function":"enableApps","class":"OCA\\Settings\\Controller\\AppSettingsController","type":"->"},{"file":"/usr/home/nextcloud/live/lib/private/AppFramework/Http/Dispatcher.php","line":133,"function":"executeController","class":"OC\\AppFramework\\Http\\Dispatcher","type":"->"},{"file":"/usr/home/nextcloud/live/lib/private/AppFramework/App.php","line":172,"function":"dispatch","class":"OC\\AppFramework\\Http\\Dispatcher","type":"->"},{"file":"/usr/home/nextcloud/live/lib/private/Route/Router.php","line":298,"function":"main","class":"OC\\AppFramework\\App","type":"::"},{"file":"/usr/home/nextcloud/live/lib/base.php","line":1047,"function":"match","class":"OC\\Route\\Router","type":"->"},{"file":"/usr/home/nextcloud/live/index.php","line":36,"function":"handleRequest","class":"OC","type":"::"}],"File":"/usr/home/nextcloud/live/3rdparty/doctrine/dbal/src/Driver/PDO/Exception.php","Line":26,"Previous":{"Exception":"PDOException","Message":"SQLSTATE[42704]: Undefined object: 7 ERROR:  index \"primary\" does not exist","Code":"42704","Trace":[{"file":"/usr/home/nextcloud/live/3rdparty/doctrine/dbal/src/Driver/PDO/Connection.php","line":82,"function":"query","class":"PDO","type":"->"},{"file":"/usr/home/nextcloud/live/3rdparty/doctrine/dbal/src/Connection.php","line":1062,"function":"query","class":"Doctrine\\DBAL\\Driver\\PDO\\Connection","type":"->"},{"file":"/usr/home/nextcloud/live/lib/private/DB/Connection.php","line":261,"function":"executeQuery","class":"Doctrine\\DBAL\\Connection","type":"->"},{"file":"/usr/home/nextcloud/live/3rdparty/doctrine/dbal/src/Connection.php","line":1809,"function":"executeQuery","class":"OC\\DB\\Connection","type":"->"},{"file":"/usr/home/nextcloud/live/lib/private/DB/Migrator.php","line":162,"function":"query","class":"Doctrine\\DBAL\\Connection","type":"->"},{"file":"/usr/home/nextcloud/live/lib/private/DB/Migrator.php","line":69,"function":"applySchema","class":"OC\\DB\\Migrator","type":"->"},{"file":"/usr/home/nextcloud/live/lib/private/DB/Connection.php","line":582,"function":"migrate","class":"OC\\DB\\Migrator","type":"->"},{"file":"/usr/home/nextcloud/live/lib/private/DB/MigrationService.php","line":454,"function":"migrateToSchema","class":"OC\\DB\\Connection","type":"->"},{"file":"/usr/home/nextcloud/live/lib/private/DB/MigrationService.php","line":408,"function":"migrateSchemaOnly","class":"OC\\DB\\MigrationService","type":"->"},{"file":"/usr/home/nextcloud/live/lib/private/Installer.php","line":153,"function":"migrate","class":"OC\\DB\\MigrationService","type":"->"},{"file":"/usr/home/nextcloud/live/apps/settings/lib/Controller/AppSettingsController.php","line":451,"function":"installApp","class":"OC\\Installer","type":"->"},{"file":"/usr/home/nextcloud/live/lib/private/AppFramework/Http/Dispatcher.php","line":225,"function":"enableApps","class":"OCA\\Settings\\Controller\\AppSettingsController","type":"->"},{"file":"/usr/home/nextcloud/live/lib/private/AppFramework/Http/Dispatcher.php","line":133,"function":"executeController","class":"OC\\AppFramework\\Http\\Dispatcher","type":"->"},{"file":"/usr/home/nextcloud/live/lib/private/AppFramework/App.php","line":172,"function":"dispatch","class":"OC\\AppFramework\\Http\\Dispatcher","type":"->"},{"file":"/usr/home/nextcloud/live/lib/private/Route/Router.php","line":298,"function":"main","class":"OC\\AppFramework\\App","type":"::"},{"file":"/usr/home/nextcloud/live/lib/base.php","line":1047,"function":"match","class":"OC\\Route\\Router","type":"->"},{"file":"/usr/home/nextcloud/live/index.php","line":36,"function":"handleRequest","class":"OC","type":"::"}],"File":"/usr/home/nextcloud/live/3rdparty/doctrine/dbal/src/Driver/PDO/Connection.php","Line":82}},"message":"could not enable apps","exception":[],"CustomMessage":"could not enable apps"},"id":"63fa44db94226"}

Additional info

No response

solracsf commented 1 year ago

Is this bug present after an update or on a fresh install?

skid9000 commented 1 year ago

I never tried to enable user_ldap before so i can't tell if this bug existed in a previous major update, sorry :/

solracsf commented 1 year ago

Ok but this isn't a fresh install, right? Just to understand if this is a migration error or a codebase error.

skid9000 commented 1 year ago

Yes, this isn't a fresh install.

solracsf commented 1 year ago

Thanks. Just to complete the report, please execute :

./occ db:add-missing-indices
./occ db:add-missing-columns
./occ db:add-missing-primary-keys

And report back please if this helps.

skid9000 commented 1 year ago

Thanks, but it dosen't seem to work :/

 nextcloud@nextcloud  ~/live  ./occ db:add-missing-indices
./occ db:add-missing-columns
./occ db:add-missing-primary-keys

Check indices of the share table.
Check indices of the filecache table.
Check indices of the twofactor_providers table.
Check indices of the login_flow_v2 table.
Check indices of the whats_new table.
Check indices of the cards table.
Check indices of the cards_properties table.
Check indices of the calendarobjects_props table.
Check indices of the schedulingobjects table.
Check indices of the oc_properties table.
Check indices of the oc_jobs table.
Check indices of the oc_direct_edit table.
Check indices of the oc_preferences table.
Check indices of the oc_mounts table.
Done.
Check columns of the comments table.
Done.
Check primary keys.
Done.
 nextcloud@nextcloud  ~/live  ./occ app:enable user_ldap  
An exception occurred while executing a query: SQLSTATE[42704]: Undefined object: 7 ERROR:  index "primary" does not exist

Nihlus commented 1 year ago

I have the same issue, also on an existing install. This is an old instance that I've been upgrading ever since the owncloud days, so it's been through a fair amount of migrations and updates. I tried running the various db:add-missing commands to no effect.

joshtrichards commented 1 year ago

Can you provide the output of:

NC_debug=1 ./occ migrations:status user_ldap

Though this almost looks like it's coming from the app management not user_ldap migrations itself.

Do you have any problem enabling other apps?
Anything unusual about this environment like was it converted from a different database at some point?

Nihlus commented 1 year ago

Sorry about the late reply, busy days. This is the output I get.

jarl@nextcloud1:/var/www/nextcloud$ sudo -u www-data NC_debug=1 ./occ migrations:status user_ldap

In MigrationService.php line 498:

  Migration step 'OCA\User_LDAP\Migration\Version1010Date20200630192842' is unknown  

migrations:status <app>

I can enable other apps just fine.

I think the most unusual thing about this instance is that it's been continuously upgraded, migrated, and moved all the way from the ownCloud days, though I can't recall if I've wiped and recreated the database at any point during these years. I believe I did so when transitioning from ownCloud to Nextcloud, but I don't know for sure.

joshtrichards commented 1 year ago

Intriguing. It'll be interesting to see if the OP (@skid9000) gets something similar.

What's the contents of your apps/user_ldap/lib/Migration folder?

skid9000 commented 1 year ago

20230909_17-11-19

joshtrichards commented 1 year ago

Something is causing some of the classes to not load for both of you.

Any weird file permissions or ownership on those files or the parent directory? (should be the same as all your other NC files hopefully)
The contents of Version1010Date2020630192842.php should be this (hopefully): https://github.com/nextcloud/server/blob/stable25/apps/user_ldap/lib/Migration/Version1010Date20200630192842.php

skid9000 commented 1 year ago

The parent dir is 755 and files are 644. Dir and files are owned by nextcloud user and www-data group.
As for the contents of Version1010Date2020630192842.php it's identical.

Nihlus commented 1 year ago

Yep, same here - contents are identical, and this is the folder listing:

jarl@nextcloud1:/var/www/nextcloud$ sudo -u www-data ls -la apps/user_ldap/lib/Migration
total 139
drwxr-xr-x  2 www-data www-data   17 Sep  9 16:16 .
drwxr-xr-x 16 www-data www-data   42 Sep  9 16:16 ..
-rw-r--r--  1 www-data www-data 2105 Sep  9 16:16 GroupMappingMigration.php
-rw-r--r--  1 www-data www-data 1879 Sep  9 16:16 RemoveRefreshTime.php
-rw-r--r--  1 www-data www-data 1650 Sep  9 16:16 SetDefaultProvider.php
-rw-r--r--  1 www-data www-data 1525 Sep  9 16:16 UnsetDefaultProvider.php
-rw-r--r--  1 www-data www-data 1146 Sep  9 16:16 UUIDFixGroup.php
-rw-r--r--  1 www-data www-data 2948 Sep  9 16:16 UUIDFixInsert.php
-rw-r--r--  1 www-data www-data 1676 Sep  9 16:16 UUIDFix.php
-rw-r--r--  1 www-data www-data 1141 Sep  9 16:16 UUIDFixUser.php
-rw-r--r--  1 www-data www-data 3447 Sep  9 16:16 Version1010Date20200630192842.php
-rw-r--r--  1 www-data www-data 4838 Sep  9 16:16 Version1120Date20210917155206.php
-rw-r--r--  1 www-data www-data 9114 Sep  9 16:16 Version1130Date20211102154716.php
-rw-r--r--  1 www-data www-data 2263 Sep  9 16:16 Version1130Date20220110154717.php
-rw-r--r--  1 www-data www-data 2928 Sep  9 16:16 Version1130Date20220110154718.php
-rw-r--r--  1 www-data www-data 1679 Sep  9 16:16 Version1130Date20220110154719.php
-rw-r--r--  1 www-data www-data 3402 Sep  9 16:16 Version1141Date20220323143801.php

Nihlus commented 1 year ago

Oh, there is one unusual thing about my setup that I forgot to mention - the entire installation directory is on an NFS share, with the data directory stored separately.

jarl@nextcloud1:~$ mount
[...]
192.168.0.25:/mnt/bones-primary/nextcloud on /mnt/nextcloud type nfs4 (rw,relatime,vers=4.2,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.0.70,local_lock=none,addr=192.168.0.25,_netdev)
192.168.0.25:/mnt/bones-primary/sites/jarl/nextcloud on /var/www/nextcloud type nfs4 (rw,relatime,sync,vers=4.2,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.0.70,local_lock=none,addr=192.168.0.25,_netdev)
[...]

skid9000 commented 1 year ago

Everything is stored locally on my side, so i doubt NFS is the problem here.

joshtrichards commented 1 year ago

Unfortunately, in hindsight, you're both getting the Migration step ... is unknown error simply due to the user_ldap app never fully getting enabled (due to the original An exception occurred while executing a query: SQLSTATE[42704]: Undefined object: 7 ERROR: index "primary" does not exist error). So, unfortunately, the inability to check the db migration status is a red herring.

I think I'm getting closer to the real culprit in any case: doctrine/dbal#2925 and is similar to #30539. It was addressed in #30568 though.

I'm currently stumped regarding why you two are impacted still.

@Nihlus Are you running PostgreSQL as well?

skid9000 commented 1 year ago

Since i made this issue i'm now running PostgreSQL 15.4 and NC 27.0.2 (w/php 8.2), i don't know if that information could help you out.

(Back then i was running PostgreSQL 14 and NC 25.0.4.1 (w/php 8.1) iirc)

Nihlus commented 1 year ago

@joshtrichards Yep, running Postgres 14.7-1.pgdg110+1. My PHP version is 7.4.

joshtrichards commented 1 month ago

It's odd that there is an empty version string for your user_ldap app installation. It should contain what is in user_ldap/appinfo/info.xml:

Disabled:
  - bruteforcesettings: 2.3.0
  - federation: 1.3.0
  - files_versions: 1.16.0
  - firstrunwizard: 2.2.1
  - gpxedit: 0.0.14
  - gpxpod: 5.0.4
  - keeweb: 0.6.10
  - sharebymail: 1.3.0
  - survey_client: 1.1.0
  - suspicious_login
  - user_ldap

Any chance you at one point ran an Release Candidate (RC) [particularly 23.0.1RC1, 22.2.4RC1] against this installation?

skid9000 commented 1 month ago

Any chance you at one point ran an Release Candidate (RC) [particularly 23.0.1RC1, 22.2.4RC1] against this installation?

I don't think so, i always stayed on stable branch and updated through the nextcloud updater, so my instance should never have seen any RC version.

nextcloud / server