Adding unfollow action in ism to invoke stop replication for ccr

[aggarwalShivani]

Issue #, if available: #726

This is to add support for unfollow feature in ism. It depends on two PRs already raised and under-review in common-utils project as well as CCR project. Detailed information about the proposed solution is explained there.

Description of changes:

1. Imported replication utilities from common-utils project
2. Added a new action "unfollow" for ism policies. It invokes the stop-replication utility from common-utils.
3. Added one integration test - (with installation of ccr plugin in testClusters.integTest )

CheckList:

• [x] Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license. For more information on following Developer Certificate of Origin and signing off your commits, please check here.

[aggarwalShivani]

Hi,

1. As the other prior two PRs in common-utils and ccr are not yet merged, the checks on CI would not go through. Also, in build.gradle, locally placed artifacts for these projects are being used. I shall remove those changes once the artifacts are merged and available in repo.
2. I've also added a UT (AttemptUnfollowStepsTest.kt) - but that is WIP as its not working correctly, as I've discussed with @bowenlan-amzn. I've tried two approaches to mock the response value, by using mockStatic (to mock ReplicationInterface) and mock spy (to mock AttemptUnfollowStep.performStopAction() alone) but havent been successful. Would like to hear if there any suggestions in this area.

Looking forward for the review, Thanks!

[aggarwalShivani]

Hi Reviewers, As suggested by @bowenlan-amzn, I'm sharing some details on how one can bring up the environment with ccr and ism for this use-case. I've done this for a linux environment.

Environment setup details

1. Setup Opensearch

$ git clone https://github.com/opensearch-project/OpenSearch.git 
$ cd Opensearch
### Ensure JAVA_HOME, PATH are set. 
$ ./gradlew assemble
$ export OPENSEARCH_HOME=<opensearch-home-dir>   ## where you wish to setup Opensearch
$ export OPENSEARCH_BUILD=distribution/archives/linux-tar/build/install/opensearch-$(./gradlew properties -q | grep -E '^version:' | awk '{print $2}')
$ cp -Rf $OPENSEARCH_BUILD/* $OPENSEARCH_HOME

(Above steps can be avoided if you already have opensearch setup) Installation of plugins:

 $ cd $OPENSEARCH_HOME
 $ ./bin/opensearch-plugin install <location-of-opensearch-job-scheduler-3.0.0.0-SNAPSHOT.zip>
 $ ./bin/opensearch-plugin install <location-of-opensearch-cross-cluster-replication-3.0.0.0-SNAPSHOT.zip>
 $ ./bin/opensearch-plugin install <location-of-opensearch-index-management-3.0.0.0-SNAPSHOT.zip>

For ex. bin/opensearch-plugin install file:///home/username/ccr/opensearch-cross-cluster-replication-3.0.0.0-SNAPSHOT.zip

2. Setup Configs We need to bring up one cluster as the leader and other as the follower (that replicates the leader indices). This small snippet is a function to generate opensearch.yml config files for the two clusters. Run this from $OPENSEARCH_HOME location.

configure_opensearch() {
     role=$1
     mkdir -p ccrtest/$role/data/  mkdir -p ccrtest/$role/logs
     cp config/opensearch.yml config/opensearch-$role.yml
     echo "
     plugins.security.disabled: true
     cluster.name: ${role}Cluster
     path.data: $OPENSEARCH_HOME/ccrtest/${role}/data
     path.logs: $OPENSEARCH_HOME/ccrtest/{role}/logs

     " >> config/opensearch-{role}.yml
}

// Invoke the function twice. It generates files opensearch-leader.yml and  opensearch-follower.yml at $OPENSEARCH_HOME/config.
configure_opensearch "leader"
configure_opensearch "follower"

(This is using the default opensearch.yml file and just adding minimal required configs for CCR)

3. Run Opensearch clusters
   Leader cluster as 127.0.0.1:9200, follower as 127.0.0.1:9201 using the generated config files respectively in step 2.

4. Start Replication using CCR REST APIs Create an index in leader cluster, and setup replication in follower cluster.

5. Start ISM Create an ism policy to run the action "unfollow" on the desired indices.

For steps 3-5, I've added the steps in a shell script for convenience - test-unfollow-script.txt. (Changed the extension to .txt as .sh files cannot be uploaded here)

I hope that helps : )