joshtrichards
commented
10 months ago Duplicate of #42232
Closed kcastner closed 10 months ago
joshtrichards
commented
10 months ago Duplicate of #42232
jangatzke
commented
10 months ago I am not sure if this is really a duplicate of #42232. I have deleted the duplicates from the database and was able to enable the LDAP app. But now this error occurs every time a user logs in for the first time. The users see an error 500 after successful login. After reloading the page, Nextcloud works fine and the users can access their files.
nextcloud.log:
,"message":"An exception occurred while executing a query: SQLSTATE[23505]: Unique violation: 7 ERROR: duplicate key value violates unique constraint \"user_ldap_membership_unique\"\nDETAIL: Key (groupid, userid)=(SQL-SPS_Error_Test-Read, FA67CEC4-4834-40B4-B440-2111B006933C) already exists.","exception":{},"CustomMessage":"An exception occurred while executing a query: SQLSTATE[23505]: Unique violation: 7 ERROR: duplicate key value violates unique constraint \"user_ldap_membership_unique\"\nDETAIL: Key (groupid, userid)=(username, FA67CEC4-4834-40B4-B440-2111B006933C) already exists."}}PSQL log:
2024-01-04 13:42:40.656 CET [2891499] ncadmin@nextcloud_db DETAIL: Key (groupid, userid)=(username, FA67CEC4-4834-40B4-B440-2111B006933C) already exists.
2024-01-04 13:42:40.656 CET [2891499] ncadmin@nextcloud_db STATEMENT: INSERT INTO "oc_ldap_group_membership" ("groupid", "userid") VALUES($1, $2)
The user in the log is not the user who logged in!
kcastner
commented
10 months ago @jangatzke i wrote in the issue @joshtrichards mentioned - i did a recheck, there are other tables related to it. till now there is no solution. the workaround is as you sayed to relog :D
my current error log looks like this - im not sure if this will help out; the collabora errors are related due to update checks. due to the 25MB file limit at github, the log is uploaded to my nextcloud.
come-nc
commented
10 months ago Reopening as this one is not from the migration but at login. According to https://github.com/nextcloud/server/issues/42232#issuecomment-1879464658 the migration went fine and the new table exists.
Have to figure out why login tries to insert a duplicated groupmembership.
come-nc
commented
10 months ago @kcastner So you only get the problem once for each user?
Can you reproduce it easily so that you would be able to provide information from both before and after the problem about the LDAP and DB content?
kcastner
commented
10 months ago @kcastner So you only get the problem once for each user?
An LDAP user logs in and the first thing he gets is a 500 error. As soon as the user presses F5 and logs in again, they can use all services.
This process can be reproduced after logout, around 20 minutes later.
Unfortunately I no longer have any backup from the error. The LDAP structure has remained the same. No new users have been created in the meantime. I use Univention Corporate Server as LDAP provider.
I can't notice any differences in the database after an LDAP login. Which tables should specifically be affected? The membership table still contains the entries that it contained before login.
jangatzke
commented
10 months ago @kcastner So you only get the problem once for each user?
An LDAP user logs in and the first thing he gets is a 500 error. As soon as the user presses F5 and logs in again, they can use all services.
This process can be reproduced after logout, around 20 minutes later.
Unfortunately I no longer have any backup from the error. The LDAP structure has remained the same. No new users have been created in the meantime. I use Univention Corporate Server as LDAP provider.
I can't notice any differences in the database after an LDAP login. Which tables should specifically be affected? The membership table still contains the entries that it contained before login.
Are yo sure you can reproduce the error with the same user after logout? I just tested it and got no error.
kcastner
commented
10 months ago Are yo sure you can reproduce the error with the same user after logout? I just tested it and got no error.
As I sayed , after round about 20 minutes the error will be back. Tried this again - I got 500 again with ID: HEy7dKu79u4705EZNXi2
My serverlog shows the following with this ID:
come-nc
commented
10 months ago Could you apply this patch, retrigger the problem and provide me the new log lines produced:
-Subproject commit a71bd8af76fdcfad78c865d1c60f6dde6e24f1dd
+Subproject commit 9e98dce036c0c2654f1ce33387f5d3e73488349e
diff --git a/apps/user_ldap/lib/LoginListener.php b/apps/user_ldap/lib/LoginListener.php
index ac5b32635c8..aaca7e71a1f 100644
--- a/apps/user_ldap/lib/LoginListener.php
+++ b/apps/user_ldap/lib/LoginListener.php
@@ -76,6 +76,22 @@ class LoginListener implements IEventListener {
);
$groupMemberships = array_combine($knownGroups, $groupMemberships);
$actualGroups = $this->groupBackend->getUserGroups($userId);
+ $this->logger->error(
+ 'Actual groups for {user}: {groups}',
+ [
+ 'app' => 'user_ldap',
+ 'user' => $userId,
+ 'groups' => json_encode($actualGroups),
+ ]
+ );
+ $this->logger->error(
+ 'Known groups for {user}: {groups}',
+ [
+ 'app' => 'user_ldap',
+ 'user' => $userId,
+ 'groups' => json_encode($knownGroups),
+ ]
+ );
$newGroups = array_diff($actualGroups, $knownGroups);
$oldGroups = array_diff($knownGroups, $actualGroups);
@@ -92,6 +108,14 @@ class LoginListener implements IEventListener {
);
continue;
}
+ $this->logger->info(
+ __CLASS__ . ' – Will add {user} to {group}',
+ [
+ 'app' => 'user_ldap',
+ 'user' => $userId,
+ 'group' => $groupId
+ ]
+ );
$this->groupMembershipMapper->insert(GroupMembership::fromParams(['groupid' => $groupId,'userid' => $userId]));
$this->groupBackend->addRelationshipToCaches($userId, null, $groupId);
$this->dispatcher->dispatchTyped(new UserAddedEvent($groupObject, $userObject));@come-nc i edited the file:
The new ID is: L3dwdwi8TnUz6lOBFPPz
come-nc
commented
10 months ago Interesting, so the problem is $this->groupBackend->getUserGroups($userId); returns a duplicated value.
Do you have several LDAP servers connected, with the same group?
kcastner
commented
10 months ago For the LDAP connection, I specify the user group that is allowed to connect (Nextcloud). I then specified which groups there should be. Here is Nextcloud and NextcloudAdmin deposited.
The logged in user is only in the Nextcloud group
come-nc
commented
10 months ago For the LDAP connection, I specify the user group that is allowed to connect (Nextcloud). I then specified which groups there should be. Here is Nextcloud and NextcloudAdmin deposited.
The logged in user is only in the Nextcloud group
My question is how many LDAP configuration do you have? If you do "occ ldap:show-config" How many configurations do you see? Only one?
If you have only one configuration I do not see yet where the duplicated group can come from as the method is supposed to run array_unique on the array before returning.
kcastner
commented
10 months ago My question is how many LDAP configuration do you have?
I only have 1 - as is see at Nextcloud-Admin-Panel.
Via command I can see 2.
+-------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------+
| Configuration | s01 |
+-------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------+
| hasMemberOfFilterSupport | 1 |
| homeFolderNamingRule | |
| lastJpegPhotoLookup | 0 |
| ldapAdminGroup | |
| ldapAgentName | uid={{ REDUCTED }},CN=users,DC=int,DC=kcastner,DC=de |
| ldapAgentPassword | *** |
| ldapAttributeAddress | |
| ldapAttributeBiography | |
| ldapAttributeFediverse | |
| ldapAttributeHeadline | |
| ldapAttributeOrganisation | |
| ldapAttributePhone | |
| ldapAttributeRole | |
| ldapAttributeTwitter | |
| ldapAttributeWebsite | |
| ldapAttributesForGroupSearch | |
| ldapAttributesForUserSearch | |
| ldapBackgroundHost | |
| ldapBackgroundPort | |
| ldapBackupHost | |
| ldapBackupPort | |
| ldapBase | DC=int,DC=kcastner,DC=de |
| ldapBaseGroups | DC=int,DC=kcastner,DC=de |
| ldapBaseUsers | DC=int,DC=kcastner,DC=de |
| ldapCacheTTL | 600 |
| ldapConfigurationActive | 1 |
| ldapConnectionTimeout | 15 |
| ldapDefaultPPolicyDN | |
| ldapDynamicGroupMemberURL | |
| ldapEmailAttribute | mail |
| ldapExperiencedAdmin | 0 |
| ldapExpertUUIDGroupAttr | |
| ldapExpertUUIDUserAttr | |
| ldapExpertUsernameAttr | |
| ldapExtStorageHomeAttribute | |
| ldapGidNumber | gidNumber |
| ldapGroupDisplayName | cn |
| ldapGroupFilter | (&(|(objectclass=univentionGroup))(|(cn=NextCloud))) |
| ldapGroupFilterGroups | NextCloud |
| ldapGroupFilterMode | 0 |
| ldapGroupFilterObjectclass | univentionGroup |
| ldapGroupMemberAssocAttr | gidNumber |
| ldapHost | int.kcastner.de |
| ldapIgnoreNamingRules | |
| ldapLoginFilter | (&(&(|(objectclass=person))(|(memberof=cn=NextCloud,cn=groups,dc=int,dc=kcastner,dc=de)))(|(uid=%uid)(|(mailPrimaryAddress=%uid)(mail=%uid)))) |
| ldapLoginFilterAttributes | |
| ldapLoginFilterEmail | 1 |
| ldapLoginFilterMode | 0 |
| ldapLoginFilterUsername | 1 |
| ldapMatchingRuleInChainState | unknown |
| ldapNestedGroups | 0 |
| ldapOverrideMainServer | |
| ldapPagingSize | 500 |
| ldapPort | 7389 |
| ldapQuotaAttribute | |
| ldapQuotaDefault | |
| ldapTLS | 0 |
| ldapUserAvatarRule | default |
| ldapUserDisplayName | displayname |
| ldapUserDisplayName2 | |
| ldapUserFilter | (&(|(objectclass=person))(|(memberof=cn=NextCloud,cn=groups,dc=int,dc=kcastner,dc=de))) |
| ldapUserFilterGroups | NextCloud |
| ldapUserFilterMode | 0 |
| ldapUserFilterObjectclass | person |
| ldapUuidGroupAttribute | auto |
| ldapUuidUserAttribute | auto |
| markRemnantsAsDisabled | 0 |
| turnOffCertCheck | 0 |
| turnOnPasswordChange | 0 |
| useMemberOfToDetectMembership | 1 |
+-------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------+
+-------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------+
| Configuration | s02 |
+-------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------+
| hasMemberOfFilterSupport | 1 |
| homeFolderNamingRule | |
| lastJpegPhotoLookup | 0 |
| ldapAdminGroup | |
| ldapAgentName | uid={{ REDUCTED }},CN=users,DC=int,DC=kcastner,DC=de |
| ldapAgentPassword | *** |
| ldapAttributeAddress | |
| ldapAttributeBiography | |
| ldapAttributeFediverse | |
| ldapAttributeHeadline | |
| ldapAttributeOrganisation | |
| ldapAttributePhone | |
| ldapAttributeRole | |
| ldapAttributeTwitter | |
| ldapAttributeWebsite | |
| ldapAttributesForGroupSearch | |
| ldapAttributesForUserSearch | |
| ldapBackgroundHost | |
| ldapBackgroundPort | |
| ldapBackupHost | |
| ldapBackupPort | |
| ldapBase | DC=int,DC=kcastner,DC=de |
| ldapBaseGroups | DC=int,DC=kcastner,DC=de |
| ldapBaseUsers | DC=int,DC=kcastner,DC=de |
| ldapCacheTTL | 600 |
| ldapConfigurationActive | 1 |
| ldapConnectionTimeout | 15 |
| ldapDefaultPPolicyDN | |
| ldapDynamicGroupMemberURL | |
| ldapEmailAttribute | mail |
| ldapExperiencedAdmin | 0 |
| ldapExpertUUIDGroupAttr | |
| ldapExpertUUIDUserAttr | |
| ldapExpertUsernameAttr | |
| ldapExtStorageHomeAttribute | |
| ldapGidNumber | gidNumber |
| ldapGroupDisplayName | cn |
| ldapGroupFilter | (&(|(objectclass=univentionGroup))(|(cn=NextCloud))) |
| ldapGroupFilterGroups | NextCloud |
| ldapGroupFilterMode | 0 |
| ldapGroupFilterObjectclass | univentionGroup |
| ldapGroupMemberAssocAttr | gidNumber |
| ldapHost | dc02.int.kcastner.de |
| ldapIgnoreNamingRules | |
| ldapLoginFilter | (&(&(|(objectclass=person))(|(memberof=cn=NextCloud,cn=groups,dc=int,dc=kcastner,dc=de)))(|(uid=%uid)(|(mailPrimaryAddress=%uid)(mail=%uid)))) |
| ldapLoginFilterAttributes | |
| ldapLoginFilterEmail | 1 |
| ldapLoginFilterMode | 0 |
| ldapLoginFilterUsername | 1 |
| ldapMatchingRuleInChainState | unknown |
| ldapNestedGroups | 0 |
| ldapOverrideMainServer | |
| ldapPagingSize | 500 |
| ldapPort | 7389 |
| ldapQuotaAttribute | |
| ldapQuotaDefault | |
| ldapTLS | 0 |
| ldapUserAvatarRule | default |
| ldapUserDisplayName | displayname |
| ldapUserDisplayName2 | |
| ldapUserFilter | (&(|(objectclass=person))(|(memberof=cn=NextCloud,cn=groups,dc=int,dc=kcastner,dc=de))) |
| ldapUserFilterGroups | NextCloud |
| ldapUserFilterMode | 0 |
| ldapUserFilterObjectclass | person |
| ldapUuidGroupAttribute | auto |
| ldapUuidUserAttribute | auto |
| markRemnantsAsDisabled | 0 |
| turnOffCertCheck | 0 |
| turnOnPasswordChange | 0 |
| useMemberOfToDetectMembership | 1 |
+-------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------+Here is a screenshot of Admin UI:
Should I delete the second config?
come-nc
commented
10 months ago It’s crazy that you do not see both config in the UI. Maybe it’s something specific to univention, I don’t know.
But at least that confirms the problem comes from multiple config. Could you try this fix:
diff --git a/apps/user_ldap/lib/Group_Proxy.php b/apps/user_ldap/lib/Group_Proxy.php
index 73daf4fdd77..7410aa6a6e9 100644
--- a/apps/user_ldap/lib/Group_Proxy.php
+++ b/apps/user_ldap/lib/Group_Proxy.php
@@ -178,7 +178,7 @@ class Group_Proxy extends Proxy implements \OCP\GroupInterface, IGroupLDAP, IGet
}
}
- return $groups;
+ return array_values(array_unique($groups));
}
/**Maybe it’s something specific to univention
i tried to figure it out - it doesnt matter which config is active. if i disable on of these via occ ldap:set-config XXX ldapConfigurationActive 0 the login will work without any problem.
if both configs are active, the problem persists.
after activating both configs, i applied your fix. the login will work again as usually.
now my question: idk how/why there are two configs set. what about deleting one? my dns-setup is correct - all DC's can be found there.
if you merge this fix to public, other instances (not mine) may have real multiple LDAP's with same group name - as i know, thats not supported. should we take care of it?
come-nc
commented
10 months ago now my question: idk how/why there are two configs set. what about deleting one? my dns-setup is correct - all DC's can be found there.
Yes I think you can delete one, but you need to figure out which one.
One has host dc02.int.kcastner.de and one has int.kcastner.de, if you know which host is better suited to be used delete the other one.
if you merge this fix to public, other instances (not mine) may have real multiple LDAP's with same group name - as i know, thats not supported. should we take care of it?
I think it is supported to merge groups accross LDAPs, at least that is how it worked until now so I will merge this fix to restore previous behavior.
Thank you for your tests and your patience!
⚠️ This issue respects the following points: ⚠️
Bug description
After login via a LDAP based user, i get this error: SQLSTATE[23000]: Integrity constraint violation: 1062 Duplicate entry 'NextCloud-da0f4174-c707-103c-89a4-dfb392a94239' for key 'user_ldap_membership_unique'
Steps to reproduce
Expected behavior
Error message should not appear after successful login
Installation method
None
Nextcloud Server version
26
Operating system
None
PHP engine version
None
Web server
None
Database engine version
None
Is this bug present after an update or on a fresh install?
None
Are you using the Nextcloud Server Encryption module?
None
What user-backends are you using?
Configuration report
List of activated Apps
Nextcloud Signing status
Nextcloud Logs
Additional info
nextcloud (1).log