search

ForgeRock / forgeops

ForgeRock platform assets for Kubernetes deployment. Contains the files you need to build your own Docker images and to deploy the ForgeRock Identity Platform on Kubernetes clusters.
Other
165 stars 243 forks source link

Replication #639

Closed daparthi001 closed 4 years ago

daparthi001 commented 4 years ago

https://github.com/ForgeRock/forgeops/blob/master/docker/6.5/ds-base/bootstrap/setup.sh

@eliasp @aldaris @wstrange @laurentvaills Do i need to chnage the echo "127.0.0.1 dsrs1.example.com dsrs2.example.com" >>/etc/hosts

dsrs1 and dsrs2 for replciation to be working on the eks ?

i am seeing issues with the below error ,but in the logs it shows replication is working host id ?

Error reading data from server

userstore-1.userstore.stg.svc.cluster.local:4444. There is an error with the certificate presented by the server. Details: Connect Error: The LDAP connection has failed because an error occurred during the SSL handshake: java.security.cert.CertificateException: No name matching userstore-1.userstore.stg.svc.cluster.local found

The displayed information might not be complete because the following errors were encountered reading the configuration of the existing servers:

An error occurred connecting to the server. Details: Connect Error: dsrs2.example.com An error occurred connecting to the server. Details: Connect Error: dsrs1.example.com No replication information found.

log message : [05/Apr/2020:22:17:18 +0000] category=SYNC severity=INFORMATION msgID=206 msg=Replication server RS(ReplicationServerId[10;10]) has accepted a connection from replication server RS(ReplicationServerId[11;11]) for domain "dc=openidm,dc=example,dc=com" at userstore-1.userstore.stg.svc.cluster.local

wstrange commented 4 years ago

The use of example.com is only during the docker build process - and must not be changed. The number of servers in the replication topology is fixed at iniital install time, and can not be changed after that. This limiitation will be removed in 7.0.

If you use one of the skaffold-6.5 profiles (see the documentation on backstage) you will get a 3 way DS replication topology. Please use one of these samples.

daparthi001 commented 4 years ago

@wstrange , we are still using the helm ,replication is not syncing users .

daparthi001 commented 4 years ago

[06/Apr/2020:15:08:05 +0000] category=SYNC severity=WARNING msgID=146 msg=Directory server DS(10) at userstore-0.userstore.stgdr.svc.cluster.local/<>:38568 presented generation ID 89397 for domain "dc=<>,dc=com", but the generation ID of this replication server RS(10) is 89326. This usually indicates that one or more directory servers in the replication topology have not been initialized with the same data, and re-initialization is required [06/Apr/2020:15:08:05 +0000] category=SYNC severity=INFORMATION msgID=207 msg=Replication server RS(ReplicationServerId[10;10]) has accepted a connection from directory server DS(ReplicaId[10;10]) for domain "dc=<>,dc=com" at userstore-0.userstore.stgdr.svc.cluster.local/<>:38568 [06/Apr/2020:15:08:05 +0000] category=SYNC severity=WARNING msgID=96 msg=Directory server DS(10) has connected to replication server RS(10) for domain "dc=<>,dc=com" at <>:8989, but the generation IDs do not match, indicating that a full re-initialization is required. The local (DS) generation ID is 89397 and the remote (RS) generation ID is 89326

Total number of matching entries: 960

forgerock@userstore-0:~/bin$

Total number of matching entries: 962

forgerock@userstore-1:~/bin$

[06/Apr/2020:15:08:17 +0000] category=SYNC severity=WARNING msgID=135 msg=Replication server RS(11) ignoring update 010201714fe5cfaa00213ee711 for domain "dc=<>,dc=com" from directory server DS(11) at userstore-1.userstore.stgdr.svc.cluster.local/<>:54542 because its generation ID 89397 is different to the local generation ID 89326

daparthi001 commented 4 years ago

context is i am restoring the contents after restore replication is not working ,what will be my hostnames to reinitialize ?Can this be done on the pods ?

https://backstage.forgerock.com/knowledge/kb/article/a36616593

wstrange commented 4 years ago

You will have to exec into the pods to complete the procedure. Because of the challenge in troubleshoot DS issues in pods, we still recommend deployment in traditional VMs at this time.

daparthi001 commented 4 years ago

what is will be hostname to connect and check the replications ?

daparthi001 commented 4 years ago

forgerock@userstore-1:~/bin$ ./dsreplication status --adminUID admin --adminPassword <> --hostname userstore-1.userstore.stgdr.svc.cluster.local --port 4444 --trustAll The displayed information might not be complete because the following errors were encountered reading the configuration of the existing servers:

Error on dsrs1.example.com: An error occurred connecting to the server. Details: Connect Error: dsrs1.example.com Error on dsrs2.example.com: An error occurred connecting to the server. Details: Connect Error: dsrs2.example.com

wstrange commented 4 years ago

The dsreplication status command does not work because it is does not interpret commons expressions in configuration. This is a known issue in 6.5 docker. You must use ldap commands on cn=monitor to query status.

Please open a ticket with ForgeRock support for further assistance.

On Mon, Apr 6, 2020 at 9:37 AM daparthi001 notifications@github.com wrote:

forgerock@userstore-1:~/bin$ ./dsreplication status --adminUID admin --adminPassword password --hostname userstore-1.userstore.stgdr.svc.cluster.local --port 4444 --trustAll The displayed information might not be complete because the following errors were encountered reading the configuration of the existing servers:

Error on dsrs1.example.com: An error occurred connecting to the server. Details: Connect Error: dsrs1.example.com Error on dsrs2.example.com: An error occurred connecting to the server. Details: Connect Error: dsrs2.example.com

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/ForgeRock/forgeops/issues/639#issuecomment-609869532, or unsubscribe https://github.com/notifications/unsubscribe-auth/AADNEZGLEZ75WIYZCFG3F3LRLHZKNANCNFSM4MB7G3UQ .

daparthi001 commented 4 years ago

can you share one simple basic command if possible just to be sure ?

wstrange commented 4 years ago

https://backstage.forgerock.com/docs/forgeops/6.5/release-notes/#chap-rnotes-issues

daparthi001 commented 4 years ago

how can i reinitialise the generation ids on pods after restore ?

https://backstage.forgerock.com/knowledge/kb/article/a36616593

daparthi001 commented 4 years ago

ds-mon-jvm-garbage-collector-g1-young-generation-count: 7 ds-mon-jvm-garbage-collector-g1-young-generation-time: 127 ds-mon-jvm-garbage-collector-g1-old-generation-count: 0 ds-mon-jvm-garbage-collector-g1-old-generation-time: 0 ds-mon-domain-generation-id: 169857 ds-mon-domain-generation-id: 169857 ds-mon-domain-generation-id: 169923 ds-mon-domain-generation-id: 8408 ds-mon-domain-generation-id: 8408 ds-mon-domain-generation-id: 8408 ds-mon-domain-generation-id: 458788 ds-mon-domain-generation-id: 458788 ds-mon-domain-generation-id: 458788 ds-mon-domain-generation-id: 89326 ds-mon-domain-generation-id: 89326 ds-mon-domain-generation-id: 89397 ds-mon-domain-generation-id: 169923 ds-mon-status: Bad generation id ds-mon-domain-generation-id: 8408 ds-mon-domain-generation-id: 458788 ds-mon-domain-generation-id: 89397 ds-mon-status: Bad generation id forgerock@userstore-0:~/bin$

daparthi001 commented 4 years ago

Prepare the domain on all servers for being externally initialized. You must specify the baseDN of the data you are going to be changing, for example: $ ./dsreplication pre-external-initialization --hostname ds1.forgerock.com --port 4444 --baseDN dc=example,dc=com --adminUID admin --adminPassword password --trustAll --no-prompt

$ ./dsreplication post-external-initialization --hostname ds1.forgerock.com --port 4444 --baseDN dc=example,dc=com --adminUID admin --adminPassword password --trustAll --no-prompt

does this to commands work on th k8s restore

daparthi001 commented 4 years ago

@wstrange can i build the docker images with userstore-0 and userstore-1 or some common names making the replication work in disaster