search

OpenIdentityPlatform / OpenDJ

OpenDJ is an LDAPv3 compliant directory service, which has been developed for the Java platform, providing a high performance, highly available, and secure store for the identities managed by your organization. Its easy installation process, combined with the power of the Java platform makes OpenDJ the simplest, fastest directory to deploy and manage.
https://www.openidentityplatform.org/opendj
Other
370 stars 103 forks source link

DS Replication Syncing Entries Successfully But Not Returning Search Results With Specific Search Options On A Given Server #363

Closed b0x68 closed 3 months ago

b0x68 commented 3 months ago

Describe the bug DS Replication shows successful sync and I can do a successful ldapsearch of a user on both of our replicated directory servers, but when searching on a specific attribute, only one of the two servers return results. This issue is very intermittent.

Sometimes, after a new user is added using a Perl script, doing an ldapsearch -h ldap-1 -LLL -x -b "ou=People,dc=cs,dc=university,dc=edu" -s sub "(objectclass=posixAccount)" uid uidNumber gidNumber gecos homeDirectory loginShell |grep cn=user1 returns the expected output on 1 replicated server, but not the other. Meanwhile if I run ldapsearch -x -h ldap-{1,2} cn=user1, it returns identical output including the posixAccount objectClass and its attributes.

In other words, When we create new accounts via automated mechanisms, sometimes certain specific searches, like on objectClass=posixAccount, only returns the expected output on only 1 of 2 replicated servers.

To Reproduce Not sure how to reproduce for others as this seems local to our environment.

Expected behavior Creating users via script should produce identical output when searching on specific objectClasses or attributes.

Specs:

- OS: RHEL 7.9
          --- Server Status ---
Server Run Status:        Started
Open Connections:         124

          --- Server Details ---
Host Name:                ldap-1.cs.university.edu
Administrative Users:     cn=Manager
Installation Path:        /opt/opendj
Version:                  OpenDJ Server 4.4.11
Java Version:             11.0.12
Administration Connector: Port 4444 (LDAPS)

          --- Connection Handlers ---
Address:Port : Protocol               : State
-------------:------------------------:---------
--           : LDIF                   : Disabled
8989         : Replication (secure)   : Enabled
0.0.0.0:389  : LDAP (allows StartTLS) : Enabled
0.0.0.0:636  : LDAPS                  : Enabled
0.0.0.0:1689 : JMX                    : Disabled
0.0.0.0:8080 : HTTP                   : Disabled

          --- Data Sources ---
Base DN:                      dc=cs,dc=university,dc=edu
Backend ID:                   userRoot
Entries:                      1837
Replication:                  Enabled
Missing Changes:              <not available>
Age of Oldest Missing Change: <not available>

Additional Information -- Logs: MODIFY event logs for replication sync.

ldap-1: {"eventName":"DJ-LDAP","client":{"ip":"","port":52539},"server":{"ip":"","port":389},"request":{"protocol":"LDAP","operation":"MODIFY","connId":307889,"msgId":7,"dn":"cn=user1,ou=People,dc=cs,dc=university,dc=edu"},"transactionId":"0","response":{"status":"SUCCESSFUL","statusCode":"0","elapsedTime":5,"elapsedTimeUnits":"MILLISECONDS"},"timestamp":"2024-07-24T13:33:22.871Z","_id":"26108ba2-088f-4f11-b840-f03cfcb21542-5873109"}

ldap-2: {"eventName":"DJ-LDAP","client":{"ip":"internal","port":-1},"server":{"ip":"internal","port":-1},"request":{"protocol":"internal","operation":"MODIFY","opType":"sync","connId":-1,"msgId":227,"dn":"cn=user1,ou=People,dc=cs,dc=university,dc=edu"},"transactionId":"0","response":{"status":"SUCCESSFUL","statusCode":"0","elapsedTime":3,"elapsedTimeUnits":"MILLISECONDS"},"timestamp":"2024-07-24T13:33:22.875Z","_id":"bbd35974-b81a-45e2-8fa0-d5665e9dfa1c-6132779"

Workaround remove and add user; do the failing ldapsearch above and both servers now show the expected output.

I imagine this is not a common issue as I haven't found much on the topic, but any troubleshooting tips would be appreciated.

vharseko commented 3 months ago

cannot reproduce on version 4.6.5

$ cd /tmp

$ rm -rf opendj

$ export VERSION="$(curl -i -o - --silent https://api.github.com/repos/OpenIdentityPlatform/OpenDJ/releases/latest | grep -m1 "\"name\"" | cut -d\" -f4)"

$ echo "last release: $VERSION"
last release: 4.6.5

$ curl -L https://github.com/OpenIdentityPlatform/OpenDJ/releases/download/$VERSION/opendj-$VERSION.zip --output opendj.zip

$ unzip opendj.zip && mv opendj opendj2

$ unzip opendj.zip && mv opendj opendj1

$ opendj1/setup --sampleData 1000 -h localhost -p 1389 --ldapsPort 1636 --adminConnectorPort 4444 --enableStartTLS --generateSelfSignedCertificate --rootUserDN "cn=Directory Manager" --rootUserPassword password --baseDN dc=example,dc=com --cli --acceptLicense --no-prompt

Configuring Directory Server ..... Done.
Configuring Certificates ..... Done.
Importing Automatically-Generated Data (1000 Entries) ...... Done.
Starting Directory Server ....... Done.

To see basic server configuration status and configuration, you can launch
/private/tmp/opendj1/bin/status

$ opendj2/setup --addBaseEntry -h localhost -p 2389 --ldapsPort 2636 --adminConnectorPort 4445 --enableStartTLS --generateSelfSignedCertificate --rootUserDN "cn=Directory Manager" --rootUserPassword password --baseDN dc=example,dc=com --cli --acceptLicense --no-prompt

Configuring Directory Server ..... Done.
Configuring Certificates ..... Done.
Creating Base Entry dc=example,dc=com ..... Done.
Starting Directory Server ...... Done.

To see basic server configuration status and configuration, you can launch
/private/tmp/opendj2/bin/status

$ opendj1/bin/dsreplication enable --host1 localhost --port1 4444 --bindDN1 "cn=Directory Manager" --bindPassword1 password --replicationPort1 1390 --host2 localhost --port2 4445 --bindDN2 "cn=Directory Manager" --bindPassword2 password --replicationPort2 2390 --adminUID admin --adminPassword password --baseDN dc=example,dc=com -X -n

Establishing connections ..... Done.
Checking registration information ..... Done.
Configuring Replication port on server localhost:4444 ..... Done.
Configuring Replication port on server localhost:4445 ..... Done.
Updating replication configuration for baseDN dc=example,dc=com on server
localhost:4444 .....Done.
Updating replication configuration for baseDN dc=example,dc=com on server
localhost:4445 .....Done.
Updating registration configuration on server localhost:4444 ..... Done.
Updating registration configuration on server localhost:4445 ..... Done.
Updating replication configuration for baseDN cn=schema on server
localhost:4444 .....Done.
Updating replication configuration for baseDN cn=schema on server
localhost:4445 .....Done.
Initializing registration information on server localhost:4445 with the
contents of server localhost:4444 .....Done.
Initializing schema on server localhost:4445 with the contents of server
localhost:4444 .....Done.

Replication has been successfully enabled.  Note that for replication to work
you must initialize the contents of the base DNs that are being replicated
(use dsreplication initialize to do so).

See
/var/folders/vh/l23lzy615jl0lwq6545txvwr0000gn/T/opendj-replication-7721374193739030006.log
for a detailed log of this operation.

$ opendj1/bin/dsreplication initialize --baseDN dc=example,dc=com --adminUID admin --adminPassword password --hostSource localhost --portSource 4444 --hostDestination localhost --portDestination 4445 -X -n

Initializing base DN dc=example,dc=com with the contents from localhost:4444:
0 entries processed (0 % complete).
1002 entries processed (100 % complete).
Base DN initialized successfully.

See
/var/folders/vh/l23lzy615jl0lwq6545txvwr0000gn/T/opendj-replication-1297405480124759720.log
for a detailed log of this operation.

$ opendj1/bin/ldapsearch --port 1389  -b "ou=people,dc=example,dc=com" -s sub "(objectclass=inetOrgPerson)" uid uidNumber gidNumber gecos homeDirectory loginShell  | grep -c uid:
1000

$ opendj2/bin/ldapsearch --port 1389  -b "ou=people,dc=example,dc=com" -s sub "(objectclass=inetOrgPerson)" uid uidNumber gidNumber gecos homeDirectory loginShell  | grep -c uid:
1000

$ opendj1/bin/ldapmodify --port 1389 --bindDN "cn=Directory Manager" --bindPassword password
dn: uid=test-ldap-1,ou=people,dc=example,dc=com
objectClass: top
objectClass: inetOrgPerson
objectClass: organizationalPerson
objectClass: person
cn: test-ldap-1
uid: test-ldap-1
sn: test-ldap-1
userPassword: password

Processing ADD request for uid=test-ldap-1,ou=people,dc=example,dc=com
ADD operation successful for DN uid=test-ldap-1,ou=people,dc=example,dc=com
^C

$ opendj1/bin/ldapsearch --port 1389  -b "ou=people,dc=example,dc=com" -s sub "(objectclass=inetOrgPerson)" uid uidNumber gidNumber gecos homeDirectory loginShell  | grep -c uid:
The LDAP search request failed: 4 (Size Limit Exceeded)
Additional Information:  This search operation has sent the maximum of 1000
entries to the client
1000

$ opendj1/bin/dsconfig set-global-configuration-prop --port 4444 --bindDN "cn=Directory Manager" --bindPassword password --set size-limit:2000 --trustAll --no-prompt
$ opendj2/bin/dsconfig set-global-configuration-prop --port 4445 --bindDN "cn=Directory Manager" --bindPassword password --set size-limit:2000 --trustAll --no-prompt

$ opendj1/bin/ldapsearch --port 1389  -b "ou=people,dc=example,dc=com" -s sub "(objectclass=inetOrgPerson)" uid uidNumber gidNumber gecos homeDirectory loginShell  | grep -c uid:
1001

$ opendj2/bin/ldapsearch --port 2389  -b "ou=people,dc=example,dc=com" -s sub "(objectclass=inetOrgPerson)" uid uidNumber gidNumber gecos homeDirectory loginShell  | grep -c uid:
1001

$ opendj2/bin/ldapmodify --port 2389 --bindDN "cn=Directory Manager" --bindPassword password
dn: uid=test-ldap-2,ou=people,dc=example,dc=com
objectClass: top
objectClass: inetOrgPerson
objectClass: organizationalPerson
objectClass: person
cn: test-ldap-2
uid: test-ldap-2
sn: test-ldap-2
userPassword: password

Processing ADD request for uid=test-ldap-2,ou=people,dc=example,dc=com
ADD operation successful for DN uid=test-ldap-2,ou=people,dc=example,dc=com
^C

$ opendj1/bin/ldapsearch --port 1389  -b "ou=people,dc=example,dc=com" -s sub "(objectclass=inetOrgPerson)" uid uidNumber gidNumber gecos homeDirectory loginShell  | grep -c uid:
1002

$ opendj2/bin/ldapsearch --port 2389  -b "ou=people,dc=example,dc=com" -s sub "(objectclass=inetOrgPerson)" uid uidNumber gidNumber gecos homeDirectory loginShell  | grep -c uid:
1002

$ opendj1/bin/ldapmodify --port 1389 --bindDN "cn=Directory Manager" --bindPassword password
dn: uid=posix-ldap-1,ou=people,dc=example,dc=com
objectClass: top
objectClass: inetOrgPerson
objectClass: organizationalPerson
objectClass: person
objectClass: posixAccount
cn: posix-ldap-1
uid: posix-ldap-1
sn: posix-ldap-1
userPassword: password
homedirectory: 1
uidnumber: 1
gidnumber: 1

Processing ADD request for uid=posix-ldap-1,ou=people,dc=example,dc=com
ADD operation successful for DN uid=posix-ldap-1,ou=people,dc=example,dc=com
^C

$ opendj1/bin/ldapsearch --port 1389  -b "ou=people,dc=example,dc=com" -s sub "(objectclass=posixAccount)" uid uidNumber gidNumber gecos homeDirectory loginShell  | grep -c uid:
1

$ opendj2/bin/ldapsearch --port 2389  -b "ou=people,dc=example,dc=com" -s sub "(objectclass=posixAccount)" uid uidNumber gidNumber gecos homeDirectory loginShell  | grep -c uid:
1

$ opendj1/bin/stop-ds 
Stopping Server...
[25/июл/2024:12:05:45 +0300] category=BACKEND severity=NOTICE msgID=370 msg=The backend userRoot is now taken offline
[25/июл/2024:12:05:45 +0300] category=CORE severity=NOTICE msgID=203 msg=The Directory Server is now stopped

$ opendj2/bin/stop-ds 
Stopping Server...
[25/июл/2024:12:05:50 +0300] category=PROTOCOL severity=NOTICE msgID=277 msg=Stopped listening for new connections on Administration Connector 0.0.0.0 port 4445
[25/июл/2024:12:05:50 +0300] category=PROTOCOL severity=NOTICE msgID=277 msg=Stopped listening for new connections on LDAP Connection Handler 0.0.0.0 port 2389
[25/июл/2024:12:05:50 +0300] category=PROTOCOL severity=NOTICE msgID=277 msg=Stopped listening for new connections on LDAPS Connection Handler 0.0.0.0 port 2636
[25/июл/2024:12:05:53 +0300] category=SYNC severity=WARNING msgID=63 msg=Replication server RS(17157) at 127.0.0.1:2390 has closed the connection to this directory server DS(12128). This directory server will now try to connect to another replication server in order to receive changes for the domain "dc=example,dc=com"
[25/июл/2024:12:05:54 +0300] category=BACKEND severity=NOTICE msgID=370 msg=The backend userRoot is now taken offline
[25/июл/2024:12:05:54 +0300] category=CORE severity=NOTICE msgID=203 msg=The Directory Server is now stopped