search

marklogic-community / roxy

Deployment tool for MarkLogic applications. Also provides optional unit test and XQuery MVC structure
Other
87 stars 66 forks source link

Roxy bootstrap creates new replica database which are not configured in ml-config.xml #820

Open ArunPrakashThangavelu opened 7 years ago

ArunPrakashThangavelu commented 7 years ago

I have tried to run a bootstrap after our Marklogic upgrade 8.0-6.6 and i noticed that there are new replica forests get created on the DB. The naming format followed is -on- . We expect the forest like anonstore-content-replica-001-1 but it created as canonstore-content-replica-001-1-on-002. I have attached the ml-config for reference. Could you please help to identify the issue here.

Forest canonstore-content-001-1 already exists, not recreated.. on server1 Forest canonstore-content-replica-001-1-on-002 succesfully created on server2 as replica of canonstore-content-001-1

jamsilvia commented 7 years ago

This is a new naming convention as of Roxy 1.7.5. The replica names are now formatted:

---on- - replica-name is from your ml-config - serverP is the sequential number for the server hosting the PRIMARY forest - forest# is which forest it is on that host - serverR is the sequential number for the server hosting the replica This was done to make it more obvious where forests exist, and to make it easier for roxy to manage when multiple replicas are specified.
ArunPrakashThangavelu commented 7 years ago

do we have any option to overwrite this behavior? We have already done the forest setup in PROD and we would like to retain the replica forest names.

jamsilvia commented 7 years ago

We do not have anything built into Roxy to support an alternate naming. With the new ability for Roxy to scale out a cluster, we moved to this new naming. If you already have replicas in place, you don't need to retire them after running bootstrap with the new Roxy version. In that case, you can just remove the new replicas and let your existing replicas stay on. It would be, however, a very manual process. You would have to do the same work for each bootstrap for future versions also (there are no plans currently to support two different naming schemes).

ArunPrakashThangavelu commented 7 years ago

Actually we have explicitly configured the replica forest names for each node as below. Can you confirm that these naming format will be applied on this as well? I think its better to derive the replica naming format only if it is not defined for nodes.

service-004-2 @ml.node2 @ml.forest-data-dir service-replica-004-2 @ml.node3
jamsilvia commented 7 years ago

Can you send your ml-config? Overall, if you are using multiple forests per node, or if your forests are striped across the cluster, then the numbering gets added.

ArunPrakashThangavelu commented 7 years ago

I have attached ml-config for your reference. Please let me know if we can do anything on this to avoid the renaming of the forests.

Thanks for your feedback. ml-config.txt

jamsilvia commented 7 years ago

We can look into adding code that if you have one (and only one) replica, and the replica name is provided, then it will directly use the name. That distinction is NOT in the code right now. I can see what @grtjn thinks about adding this in. If it sounds like something that would be beneficial to deliver, I can get you a version of setup.xqy that has the update in it for 1.7.6. Otherwise, you'll have to wait for the next release.

ArunPrakashThangavelu commented 7 years ago

Yes it would be helpful if we can get this update now.

Also i would like to understand the sequence number derived for the server nodes in the cluster, For instance in my case, i have 4 nodes in the cluster and i have seen the behaviour as below. Can you please help to explain this.

node1 = -on-04 node2 = -on-01 node3 = -on-02 node4 = -on-03

grtjn commented 7 years ago

I think it would be good to have a legacy flag that turns on the old naming scheme for replica forests. This is not the first case where people tell us they can't simply switch to the new naming scheme because they are in production.

Would be nice if we could make the naming pattern configurable, but that is probably not that easy..

jamsilvia commented 7 years ago

OK - I'll work on an update this weekend. Arun - can you give me a snapshot of your forests/hosts admin page? Then I will be able to get a context for your question.

joe

ArunPrakashThangavelu commented 7 years ago

Hi Joe, I have attached the forests that are attached in our DB where the new forests got created. Also attached the hosts summary screen. screenshots.zip

jamsilvia commented 7 years ago

Sorry for the delay - I had some personal stuff I had to take care of.

So, looking at your screenshots first...

The numbering is consistent. So, the host counter is based on the ordering of the hosts in the "hosts.xml" file. If the order of your hosts in that file doesn't match the logical order, then there would be a mismatch. For example, host "itgsydapu743" is supposed to be host #3 - but it must appear as the 2nd host in the "hosts.xml" file.

Note that this is only as consistent as your hosts.xml file is across the nodes in the cluster. If your hosts are in order 12345 on host #1, but in order 54321 on host #5....your results will be different running the deployment on #1 vs #5.

Attached is a short script you can run from qconsole to see what order gets returned for your hosts.

host-order.txt

Working on an updated server.xqy file for you to put into your deployment - and I'll work with @grtjn to get it into future releases.

jamsilvia commented 7 years ago

Also - @ArunPrakashThangavelu, can you verify the host order across your cluster? It looks like the same ordering has been used by Roxy for a while now, so I'd like to know whether something has changed that may have triggered a configuration difference. My script update would name everything like it used to be - but the numbering is still based on the ordering returned from the admin:group-get-host-ids() call.

jamsilvia commented 7 years ago

@ArunPrakashThangavelu - I haven't heard back from you Re: host order. I am planning on putting in the change to allow legacy naming, but need to verify that we don't have another problem with host ordering.

Thank you!