opensearch-project / OpenSearch

🔎 Open source distributed and RESTful search engine.
https://opensearch.org/docs/latest/opensearch/index/
Apache License 2.0
9.84k stars 1.83k forks source link

Make IndexStoreListener a pluggable interface #16583

Closed jed326 closed 2 weeks ago

jed326 commented 2 weeks ago

Description

I have a use case where I have a custom file cache implementation and in order to not have dangling entries in my fc I need to implement IndexStoreListener to clean up my file cache when shards move off a node. Today that is not possible as IndexStoreListener is an @opensearch.internal interface and not pluggable, which this PR changes.

Long term I think the solution is to make the FileCache itself pluggable, but that is much more complicated and anyways would require us to expose this interface as pluggable, so I believe this is a useful intermediary step.

The other thing to note is in Node.java I am now creating the CompositeIndexStoreListener regardless of if it isSearchNode() or not as it should be up to the specific listener implementations to determine how/when to perform their action.

Check List

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license. For more information on following Developer Certificate of Origin and signing off your commits, please check here.

jed326 commented 2 weeks ago

@andrross @bugmakerrrrrr @sohami could you review this PR as it is related to #11443?

github-actions[bot] commented 2 weeks ago

:x: Gradle check result for 8c4fd86a989eb51ff3090541f1fe25417d70a85b: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

github-actions[bot] commented 2 weeks ago

:x: Gradle check result for ceeb08ab48a27c2da9225141073b31bfbafba6ac: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

github-actions[bot] commented 2 weeks ago

:x: Gradle check result for 65497a053b9ed432a3620cc865273ddcaf9693e6: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

github-actions[bot] commented 2 weeks ago

:white_check_mark: Gradle check result for 2de2403420fd5495ccd379a6d3f13256c672bb20: SUCCESS

codecov[bot] commented 2 weeks ago

Codecov Report

Attention: Patch coverage is 87.50000% with 4 lines in your changes missing coverage. Please review.

Project coverage is 72.11%. Comparing base (034bd2b) to head (ca32a2f). Report is 3 commits behind head on main.

Files with missing lines Patch % Lines
...org/opensearch/index/store/IndexStoreListener.java 80.95% 4 Missing :warning:
Additional details and impacted files ```diff @@ Coverage Diff @@ ## main #16583 +/- ## ============================================ + Coverage 72.04% 72.11% +0.07% - Complexity 65093 65102 +9 ============================================ Files 5314 5315 +1 Lines 303541 303572 +31 Branches 43921 43925 +4 ============================================ + Hits 218683 218935 +252 + Misses 66946 66672 -274 - Partials 17912 17965 +53 ```

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

sohami commented 2 weeks ago

lgtm. Will wait for others to approve as well

github-actions[bot] commented 2 weeks ago

:grey_exclamation: Gradle check result for ca32a2f85b36a99d93d757b0b01cee169a084dbf: UNSTABLE

Please review all flaky tests that succeeded after retry and create an issue if one does not already exist to track the flaky failure.