Added changes for AdmissionControl Interceptor and AdmissionControlService for RateLimiting

ajaymovva commented 9 months ago

Description

This PR consists of below functionalities

Adding core transport interceptors in network module to intercept the requests at the transport layer.
AdmissionControlInterceptor to add the new AdmissionControlHandler, and that handler will be triggered for every transport request. -The initial implementation of AdmissionControlService, which is currently used to register the admission controllers and trigger the registered admission controllers for every transport request if it is enabled.
AdmissionControlSettings -> Settings related to AdmissionController Feature.
Basic Implementation of the CPU-Based Admission Controller, which currently only tracks the number of transport calls if the admission controller is enabled.
CPUBasedAdmissionControllerSettings, which have the settings specific to the CPU-based admission controller.

Note:

This PR only have initial basic implementation of many classes. As this is the first PR for this feature we are making sure to give basic idea how admission controller is tracing the transport requests and eventually how we will reject the requests if the threshold breaches based on each admission controllers

Check List

[x] New functionality includes testing.
- [x] All tests pass
[x] New functionality has been documented.
- [x] New functionality has javadoc added
[x] Commits are signed per the DCO using --signoff
[x] Commit changes are listed out in CHANGELOG.md file (See: Changelog)

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license. For more information on following Developer Certificate of Origin and signing off your commits, please check here.

github-actions[bot] commented 9 months ago

Gradle Check (Jenkins) Run Completed with:

RESULT: FAILURE :x:
URL: https://build.ci.opensearch.org/job/gradle-check/22581/
CommitID: d9d068641303b8c38ac436cf5ae4046a6fe8b46b Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

opensearch-trigger-bot[bot] commented 9 months ago

Compatibility status:


> Task :checkCompatibility
Incompatible components: [https://github.com/opensearch-project/alerting.git, https://github.com/opensearch-project/index-management.git, https://github.com/opensearch-project/asynchronous-search.git, https://github.com/opensearch-project/sql.git, https://github.com/opensearch-project/observability.git, https://github.com/opensearch-project/common-utils.git, https://github.com/opensearch-project/reporting.git, https://github.com/opensearch-project/cross-cluster-replication.git, https://github.com/opensearch-project/notifications.git, https://github.com/opensearch-project/security-analytics.git, https://github.com/opensearch-project/performance-analyzer.git]
Compatible components: [https://github.com/opensearch-project/security.git, https://github.com/opensearch-project/anomaly-detection.git, https://github.com/opensearch-project/job-scheduler.git, https://github.com/opensearch-project/k-nn.git, https://github.com/opensearch-project/geospatial.git, https://github.com/opensearch-project/neural-search.git, https://github.com/opensearch-project/ml-commons.git, https://github.com/opensearch-project/performance-analyzer-rca.git, https://github.com/opensearch-project/opensearch-oci-object-storage.git]

BUILD SUCCESSFUL in 24m 6s

github-actions[bot] commented 9 months ago

Gradle Check (Jenkins) Run Completed with:

RESULT: FAILURE :x:
URL: https://build.ci.opensearch.org/job/gradle-check/22778/
CommitID: babe2118a96bf6780a7ddd4aa7d82a022adab21e Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

github-actions[bot] commented 9 months ago

Gradle Check (Jenkins) Run Completed with:

RESULT: FAILURE :x:
URL: https://build.ci.opensearch.org/job/gradle-check/22779/
CommitID: efbd221b93db2519c642004b3455e2c32577dc04 Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

opensearch-trigger-bot[bot] commented 9 months ago

Compatibility status:


> Task :checkCompatibility
Incompatible components: [https://github.com/opensearch-project/notifications.git, https://github.com/opensearch-project/index-management.git, https://github.com/opensearch-project/security-analytics.git, https://github.com/opensearch-project/alerting.git, https://github.com/opensearch-project/asynchronous-search.git, https://github.com/opensearch-project/common-utils.git, https://github.com/opensearch-project/cross-cluster-replication.git]
Compatible components: [https://github.com/opensearch-project/geospatial.git, https://github.com/opensearch-project/security.git, https://github.com/opensearch-project/neural-search.git, https://github.com/opensearch-project/sql.git, https://github.com/opensearch-project/job-scheduler.git, https://github.com/opensearch-project/opensearch-oci-object-storage.git, https://github.com/opensearch-project/observability.git, https://github.com/opensearch-project/k-nn.git, https://github.com/opensearch-project/anomaly-detection.git, https://github.com/opensearch-project/reporting.git, https://github.com/opensearch-project/ml-commons.git, https://github.com/opensearch-project/performance-analyzer.git, https://github.com/opensearch-project/performance-analyzer-rca.git]

BUILD SUCCESSFUL in 32m 6s

github-actions[bot] commented 9 months ago

Gradle Check (Jenkins) Run Completed with:

RESULT: FAILURE :x:
URL: https://build.ci.opensearch.org/job/gradle-check/23235/
CommitID: 06a4ff45c9f23bfc3c09c995663bbfc087b68c0c Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

github-actions[bot] commented 9 months ago

Gradle Check (Jenkins) Run Completed with:

RESULT: FAILURE :x:
URL: https://build.ci.opensearch.org/job/gradle-check/23800/
CommitID: 9a65c7c89fc1b1e46ed32d96012c4b8437901237 Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

github-actions[bot] commented 9 months ago

Compatibility status:

Checks if related components are compatible with change fa39f9a

Incompatible components

Skipped components

Compatible components

github-actions[bot] commented 9 months ago

Gradle Check (Jenkins) Run Completed with:

RESULT: FAILURE :x:
URL: https://build.ci.opensearch.org/job/gradle-check/23801/
CommitID: be089cf09d1902336075744c3272e0bc6b7fef64 Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

github-actions[bot] commented 9 months ago

Compatibility status:

Checks if related components are compatible with change be089cf

Incompatible components

Skipped components

Compatible components

github-actions[bot] commented 9 months ago

Compatibility status:

Checks if related components are compatible with change a1e6134

Incompatible components

Skipped components

Compatible components

github-actions[bot] commented 9 months ago

Gradle Check (Jenkins) Run Completed with:

RESULT: FAILURE :x:
URL: https://build.ci.opensearch.org/job/gradle-check/24284/
CommitID: a1e61345920fe48de23ec2eacd7c02e571e2b7d7 Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

github-actions[bot] commented 9 months ago

Gradle Check (Jenkins) Run Completed with:

RESULT: FAILURE :x:
URL: https://build.ci.opensearch.org/job/gradle-check/24290/
CommitID: f60c6b5d9b32bf70774a2a0daa15c995932833be Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

github-actions[bot] commented 9 months ago

Compatibility status:

Checks if related components are compatible with change f60c6b5

Incompatible components

Skipped components

Compatible components

github-actions[bot] commented 9 months ago

Compatibility status:

Checks if related components are compatible with change de4ea96

Incompatible components

Skipped components

Compatible components

github-actions[bot] commented 9 months ago

Gradle Check (Jenkins) Run Completed with:

RESULT: FAILURE :x:
URL: https://build.ci.opensearch.org/job/gradle-check/24329/
CommitID: de4ea96061702eb7b0e48f13d2d0bd128eae97c7 Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

github-actions[bot] commented 9 months ago

Compatibility status:

Checks if related components are compatible with change 25c16b6

Incompatible components

Skipped components

Compatible components

github-actions[bot] commented 9 months ago

Gradle Check (Jenkins) Run Completed with:

RESULT: UNSTABLE :grey_exclamation:

TEST FAILURES:

  1 org.opensearch.repositories.azure.AzureBlobContainerRetriesTests.testWriteLargeBlob

URL: https://build.ci.opensearch.org/job/gradle-check/24337/
CommitID: 25c16b65453c769f548556945b123632a6d32861 Please review all flaky tests that succeeded after retry and create an issue if one does not already exist to track the flaky failure.

codecov[bot] commented 9 months ago

Codecov Report

Merging #9286 (fa39f9a) into main (200ad5d) will increase coverage by 0.09%. Report is 4 commits behind head on main. The diff coverage is 92.14%.

@@             Coverage Diff              @@
##               main    #9286      +/-   ##
============================================
+ Coverage     71.12%   71.21%   +0.09%     
- Complexity    58545    58688     +143     
============================================
  Files          4859     4868       +9     
  Lines        276252   276399     +147     
  Branches      40191    40195       +4     
============================================
+ Hits         196473   196847     +374     
+ Misses        63347    63185     -162     
+ Partials      16432    16367      -65

Files	Coverage Δ
...upport/replication/TransportReplicationAction.java	`77.28% <100.00%> (-3.40%)`	:arrow_down:
...a/org/opensearch/common/network/NetworkModule.java	`92.20% <100.00%> (+0.20%)`	:arrow_up:
...rg/opensearch/common/settings/ClusterSettings.java	`92.85% <ø> (ø)`
.../indices/replication/common/ReplicationTarget.java	`86.25% <100.00%> (+7.30%)`	:arrow_up:
server/src/main/java/org/opensearch/node/Node.java	`85.28% <100.00%> (+0.09%)`	:arrow_up:
...ting/admissioncontrol/AdmissionControlService.java	`100.00% <100.00%> (ø)`
...issioncontrol/controllers/AdmissionController.java	`100.00% <100.00%> (ø)`
...g/admissioncontrol/enums/AdmissionControlMode.java	`100.00% <100.00%> (ø)`
...ng/admissioncontrol/enums/TransportActionType.java	`100.00% <100.00%> (ø)`
...ol/transport/AdmissionControlTransportHandler.java	`100.00% <100.00%> (ø)`
... and 7 more

... and 471 files with indirect coverage changes

github-actions[bot] commented 8 months ago

Compatibility status:

Checks if related components are compatible with change 7f87c0e

Incompatible components

Incompatible components: [https://github.com/opensearch-project/sql.git, https://github.com/opensearch-project/cross-cluster-replication.git, https://github.com/opensearch-project/neural-search.git]

Skipped components

Compatible components

github-actions[bot] commented 8 months ago

Compatibility status:

Checks if related components are compatible with change fc249df

Incompatible components

Incompatible components: [https://github.com/opensearch-project/sql.git, https://github.com/opensearch-project/cross-cluster-replication.git, https://github.com/opensearch-project/neural-search.git]

Skipped components

Compatible components

github-actions[bot] commented 8 months ago

Gradle Check (Jenkins) Run Completed with:

RESULT: FAILURE :x:
URL: https://build.ci.opensearch.org/job/gradle-check/24664/
CommitID: 7f87c0e6171ad0749228388379f93a5b17c3ef68 Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

github-actions[bot] commented 8 months ago

Gradle Check (Jenkins) Run Completed with:

RESULT: SUCCESS :white_check_mark:
URL: https://build.ci.opensearch.org/job/gradle-check/24667/
CommitID: fc249df4ab8af7e6bfc7b8b3db15868021904a47

github-actions[bot] commented 8 months ago

Compatibility status:

Checks if related components are compatible with change eaf67e0

Incompatible components

Incompatible components: [https://github.com/opensearch-project/sql.git, https://github.com/opensearch-project/cross-cluster-replication.git, https://github.com/opensearch-project/neural-search.git]

Skipped components

Compatible components

github-actions[bot] commented 8 months ago

Gradle Check (Jenkins) Run Completed with:

RESULT: FAILURE :x:
URL: https://build.ci.opensearch.org/job/gradle-check/24719/
CommitID: eaf67e0ac9e3d68d92cb23d4071ca2be19ede44d Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

github-actions[bot] commented 8 months ago

Compatibility status:

Checks if related components are compatible with change 63035a8

Incompatible components

Incompatible components: [https://github.com/opensearch-project/sql.git, https://github.com/opensearch-project/cross-cluster-replication.git, https://github.com/opensearch-project/neural-search.git]

Skipped components

Compatible components

github-actions[bot] commented 8 months ago

Gradle Check (Jenkins) Run Completed with:

RESULT: SUCCESS :white_check_mark:
URL: https://build.ci.opensearch.org/job/gradle-check/24758/
CommitID: 63035a80711c0648eb3457893e84728f1bb2de33

github-actions[bot] commented 8 months ago

Gradle Check (Jenkins) Run Completed with:

RESULT: UNSTABLE :grey_exclamation:

TEST FAILURES:

  1 org.opensearch.repositories.azure.AzureBlobStoreRepositoryTests.testContainerCreationAndDeletion

URL: https://build.ci.opensearch.org/job/gradle-check/25252/
CommitID: ea3105ffee6ba7374652b703ed187cf1b66c6c7d Please review all flaky tests that succeeded after retry and create an issue if one does not already exist to track the flaky failure.

github-actions[bot] commented 8 months ago

Gradle Check (Jenkins) Run Completed with:

RESULT: SUCCESS :white_check_mark:
URL: https://build.ci.opensearch.org/job/gradle-check/25256/
CommitID: 78bd4940716d897ca5e48d71fa74c8c223974a65

github-actions[bot] commented 8 months ago

Gradle Check (Jenkins) Run Completed with:

RESULT: FAILURE :x:
URL: https://build.ci.opensearch.org/job/gradle-check/25259/
CommitID: 42e8a93fc69ef0032ccc0539f01da617f3706db5 Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

github-actions[bot] commented 8 months ago

Gradle Check (Jenkins) Run Completed with:

RESULT: SUCCESS :white_check_mark:
URL: https://build.ci.opensearch.org/job/gradle-check/25279/
CommitID: f4505453acd51e9ca30e8250ef0337e99c9013aa

github-actions[bot] commented 8 months ago

Gradle Check (Jenkins) Run Completed with:

RESULT: UNSTABLE :grey_exclamation:

TEST FAILURES:

  1 org.opensearch.snapshots.CloneSnapshotIT.testCloneAfterRepoShallowSettingDisabled

URL: https://build.ci.opensearch.org/job/gradle-check/25370/
CommitID: 7b1e1173cbdf1590a1de739bbbe6ac4536165aae Please review all flaky tests that succeeded after retry and create an issue if one does not already exist to track the flaky failure.

github-actions[bot] commented 8 months ago

Gradle Check (Jenkins) Run Completed with:

RESULT: UNSTABLE :grey_exclamation:

TEST FAILURES:

  1 org.opensearch.remotestore.SegmentReplicationUsingRemoteStoreIT.testUpdateOperations
  1 org.opensearch.remotestore.RemoteStoreStatsIT.testStatsOnRemoteStoreRestore

URL: https://build.ci.opensearch.org/job/gradle-check/25372/
CommitID: 0598be041323f3d3417845c4f599301c626168e5 Please review all flaky tests that succeeded after retry and create an issue if one does not already exist to track the flaky failure.

github-actions[bot] commented 8 months ago

Gradle Check (Jenkins) Run Completed with:

RESULT: FAILURE :x:
URL: https://build.ci.opensearch.org/job/gradle-check/26282/
CommitID: 4bb7740351c623b9d05d8cd377d3a9b8e87afe09 Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

github-actions[bot] commented 8 months ago

Gradle Check (Jenkins) Run Completed with:

RESULT: FAILURE :x:
URL: https://build.ci.opensearch.org/job/gradle-check/26304/
CommitID: 06c546f634a2a6b81bb6baeb32d5c4588b51d72d Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

github-actions[bot] commented 8 months ago

Gradle Check (Jenkins) Run Completed with:

RESULT: FAILURE :x:
URL: https://build.ci.opensearch.org/job/gradle-check/26305/
CommitID: 600ec86886d5c4dfadc735102800e5b0b2622e48 Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

github-actions[bot] commented 8 months ago

Gradle Check (Jenkins) Run Completed with:

RESULT: FAILURE :x:
URL: https://build.ci.opensearch.org/job/gradle-check/26307/
CommitID: 037f6eb4827a891e5980a477b6b0cd646d5c10dc Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

github-actions[bot] commented 8 months ago

Gradle Check (Jenkins) Run Completed with:

RESULT: FAILURE :x:
URL: https://build.ci.opensearch.org/job/gradle-check/26316/
CommitID: 7934f85c2b1be5d7d7e240c6e8a3c143aae86a84 Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

github-actions[bot] commented 8 months ago

Gradle Check (Jenkins) Run Completed with:

RESULT: UNSTABLE :grey_exclamation:

TEST FAILURES:

  1 org.opensearch.indices.replication.SegmentReplicationRelocationIT.testRelocateWhileContinuouslyIndexingAndWaitingForRefresh
  1 org.opensearch.indices.replication.SegmentReplicationRelocationIT.testRelocateWhileContinuouslyIndexingAndWaitingForRefresh
  1 org.opensearch.indices.replication.SegmentReplicationRelocationIT.testPrimaryRelocation

URL: https://build.ci.opensearch.org/job/gradle-check/26318/
CommitID: 65302ac6c507df4b64b5a9183943026b72f31e4d Please review all flaky tests that succeeded after retry and create an issue if one does not already exist to track the flaky failure.

ajaymovva commented 8 months ago

Tagging @reta for the review feedback on the PR.

github-actions[bot] commented 8 months ago

Gradle Check (Jenkins) Run Completed with:

RESULT: FAILURE :x:
URL: https://build.ci.opensearch.org/job/gradle-check/26405/
CommitID: f13b873ccb2b925ab7d65e3744cf0642bec498bd Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

github-actions[bot] commented 8 months ago

Gradle Check (Jenkins) Run Completed with:

RESULT: FAILURE :x:
URL: https://build.ci.opensearch.org/job/gradle-check/26407/
CommitID: cd6446048b954b683898c179ce049c0ca7b071e1 Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

github-actions[bot] commented 8 months ago

Gradle Check (Jenkins) Run Completed with:

RESULT: UNSTABLE :grey_exclamation:

TEST FAILURES:

  1 org.opensearch.snapshots.CloneSnapshotIT.testCloneAfterRepoShallowSettingDisabled

URL: https://build.ci.opensearch.org/job/gradle-check/26433/
CommitID: 2ab38115db4f3638e4bed82d7364fa8b57910268 Please review all flaky tests that succeeded after retry and create an issue if one does not already exist to track the flaky failure.

github-actions[bot] commented 8 months ago

Gradle Check (Jenkins) Run Completed with:

RESULT: FAILURE :x:
URL: https://build.ci.opensearch.org/job/gradle-check/26899/
CommitID: 41505b5523f94b2d07a033e252699ccd98aeb4c8 Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

github-actions[bot] commented 8 months ago

Gradle Check (Jenkins) Run Completed with:

RESULT: :x:
URL: https://build.ci.opensearch.org/job/gradle-check/26965/
CommitID: b840dde8c182955df74e30da2ffcf83103076758 Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

github-actions[bot] commented 8 months ago

Gradle Check (Jenkins) Run Completed with:

RESULT: :x:
URL: https://build.ci.opensearch.org/job/gradle-check/26960/
CommitID: 87e5360631431d2f072b682fafbdbe0666a1345b Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

github-actions[bot] commented 8 months ago

Gradle Check (Jenkins) Run Completed with:

RESULT: FAILURE :x:
URL: https://build.ci.opensearch.org/job/gradle-check/27094/
CommitID: 53aa86f28db9fcb5e359b228cbcd2b878fcd2508 Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

github-actions[bot] commented 7 months ago

Gradle Check (Jenkins) Run Completed with:

RESULT: FAILURE :x:
URL: https://build.ci.opensearch.org/job/gradle-check/27756/
CommitID: 334dfc5d06cecf1e91b616f99c32a7856008d560 Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

github-actions[bot] commented 7 months ago

Gradle Check (Jenkins) Run Completed with:

RESULT: null :x:
URL: https://build.ci.opensearch.org/job/gradle-check/27766/
CommitID: 8f370e194753f55393802c41aded20cf5e211b03 Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

github-actions[bot] commented 7 months ago

Gradle Check (Jenkins) Run Completed with:

RESULT: :x:
URL: https://build.ci.opensearch.org/job/gradle-check/27765/
CommitID: ef8f38e9ac718fef656cc4685d64f411cfc40083 Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

opensearch-project / OpenSearch