BhumikaSaini-Amazon commented 10 months ago

Description

This PR adds the support for exposing stats related to Remote Translog Store (RTS) upload operations introduced in https://github.com/opensearch-project/OpenSearch/pull/9263 under the _nodes/stats/ API. We also add rollup logic to surface aggregated visibility on RTS operations at the level of indices and nodes.

A sample response with these changes would look like:

        "indices" : {
          "idx1" : {
            "translog" : {
              "operations" : 5,
              "size_in_bytes" : 970,
              "uncommitted_operations" : 5,
              "uncommitted_size_in_bytes" : 970,
              "earliest_last_modified_age" : 13684,
              "remote_store" : {
                "upload" : {
                  "total_uploads" : {
                    "started" : 44,
                    "failed" : 0,
                    "succeeded" : 44
                  },
                  "total_uploads_in_bytes" : {
                    "started" : 9988,
                    "failed" : 0,
                    "succeeded" : 9988
                  }
                }
              }
            }
          }
        }

Related Issues

8311

Check List

[x] New functionality includes testing.
- [ ] All tests pass
[x] New functionality has been documented.
- [x] New functionality has javadoc added
[x] Commits are signed per the DCO using --signoff
[x] ~Commit changes are listed out in CHANGELOG.md file (See: Changelog)~

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license. For more information on following Developer Certificate of Origin and signing off your commits, please check here.

BhumikaSaini-Amazon commented 10 months ago

TODOs that I am working on:

[x] Adding the missing javadocs
[x] Fixing the sequence of assertEquals() params in the RSS stats tests (currently the tests are asserting the actual value as the expected value)
[x] Investigating why org.opensearch.index.translog.RemoteFSTranslogTests.testSyncUpFailure fails flakily
[x] Checking IT failures
[x] Integrating changes from #8718

github-actions[bot] commented 10 months ago

Gradle Check (Jenkins) Run Completed with:

RESULT: FAILURE :x:
URL: https://build.ci.opensearch.org/job/gradle-check/21048/
CommitID: 794e497da4aa823560facf756ef63ab806235b95 Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

github-actions[bot] commented 10 months ago

Gradle Check (Jenkins) Run Completed with:

RESULT: FAILURE :x:
URL: https://build.ci.opensearch.org/job/gradle-check/21392/
CommitID: 08b051bc20bd5651abb7b52bc719f7bdd9f86a36 Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

github-actions[bot] commented 10 months ago

Gradle Check (Jenkins) Run Completed with:

RESULT: FAILURE :x:
URL: https://build.ci.opensearch.org/job/gradle-check/21420/
CommitID: 0a75cd13d87651305fc3a03da2544f1941277dfe Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

BhumikaSaini-Amazon commented 10 months ago

Moving this to draft for now while I do a PoC on integrating this with TranslogStats

github-actions[bot] commented 10 months ago

Gradle Check (Jenkins) Run Completed with:

RESULT: FAILURE :x:
URL: https://build.ci.opensearch.org/job/gradle-check/21668/
CommitID: bbd31ddea7f70622a5d7512c86f6d944cf13c079 Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

github-actions[bot] commented 10 months ago

Gradle Check (Jenkins) Run Completed with:

RESULT: FAILURE :x:
URL: https://build.ci.opensearch.org/job/gradle-check/21683/
CommitID: a2ca2cb65d9e58961e2fbe0ba98a37650b403958 Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

github-actions[bot] commented 10 months ago

Gradle Check (Jenkins) Run Completed with:

RESULT: FAILURE :x:
URL: https://build.ci.opensearch.org/job/gradle-check/21684/
CommitID: eec34421a646f92f47a853e6978547c8083746b5 Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

opensearch-trigger-bot[bot] commented 10 months ago

Compatibility status:


> Task :checkCompatibility
Incompatible components: [https://github.com/opensearch-project/index-management.git, https://github.com/opensearch-project/anomaly-detection.git, https://github.com/opensearch-project/asynchronous-search.git, https://github.com/opensearch-project/reporting.git, https://github.com/opensearch-project/geospatial.git, https://github.com/opensearch-project/performance-analyzer.git, https://github.com/opensearch-project/security-analytics.git]
Compatible components: [https://github.com/opensearch-project/security.git, https://github.com/opensearch-project/alerting.git, https://github.com/opensearch-project/sql.git, https://github.com/opensearch-project/job-scheduler.git, https://github.com/opensearch-project/common-utils.git, https://github.com/opensearch-project/observability.git, https://github.com/opensearch-project/k-nn.git, https://github.com/opensearch-project/cross-cluster-replication.git, https://github.com/opensearch-project/notifications.git, https://github.com/opensearch-project/performance-analyzer-rca.git, https://github.com/opensearch-project/neural-search.git, https://github.com/opensearch-project/ml-commons.git, https://github.com/opensearch-project/opensearch-oci-object-storage.git]

BUILD SUCCESSFUL in 37m 49s

github-actions[bot] commented 10 months ago

Gradle Check (Jenkins) Run Completed with:

RESULT: FAILURE :x:
URL: https://build.ci.opensearch.org/job/gradle-check/21686/
CommitID: b78177d1cbbc3795694c0011cecbd235a8b0aa7d Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

github-actions[bot] commented 9 months ago

Gradle Check (Jenkins) Run Completed with:

RESULT: FAILURE :x:
URL: https://build.ci.opensearch.org/job/gradle-check/22314/
CommitID: 3f30b21c7f29b2cc2a2d71c1015cf8f0d5385585 Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

opensearch-trigger-bot[bot] commented 9 months ago

Compatibility status:


> Task :checkCompatibility
Incompatible components: [https://github.com/opensearch-project/index-management.git, https://github.com/opensearch-project/asynchronous-search.git, https://github.com/opensearch-project/performance-analyzer.git]
Compatible components: [https://github.com/opensearch-project/security.git, https://github.com/opensearch-project/alerting.git, https://github.com/opensearch-project/anomaly-detection.git, https://github.com/opensearch-project/sql.git, https://github.com/opensearch-project/job-scheduler.git, https://github.com/opensearch-project/common-utils.git, https://github.com/opensearch-project/observability.git, https://github.com/opensearch-project/k-nn.git, https://github.com/opensearch-project/reporting.git, https://github.com/opensearch-project/cross-cluster-replication.git, https://github.com/opensearch-project/geospatial.git, https://github.com/opensearch-project/notifications.git, https://github.com/opensearch-project/neural-search.git, https://github.com/opensearch-project/ml-commons.git, https://github.com/opensearch-project/performance-analyzer-rca.git, https://github.com/opensearch-project/security-analytics.git, https://github.com/opensearch-project/opensearch-oci-object-storage.git]

BUILD SUCCESSFUL in 38m 57s

opensearch-trigger-bot[bot] commented 9 months ago

Compatibility status:


> Task :checkCompatibility
Incompatible components: [https://github.com/opensearch-project/security.git, https://github.com/opensearch-project/alerting.git, https://github.com/opensearch-project/anomaly-detection.git, https://github.com/opensearch-project/asynchronous-search.git, https://github.com/opensearch-project/index-management.git, https://github.com/opensearch-project/sql.git, https://github.com/opensearch-project/job-scheduler.git, https://github.com/opensearch-project/common-utils.git, https://github.com/opensearch-project/observability.git, https://github.com/opensearch-project/reporting.git, https://github.com/opensearch-project/k-nn.git, https://github.com/opensearch-project/geospatial.git, https://github.com/opensearch-project/cross-cluster-replication.git, https://github.com/opensearch-project/notifications.git, https://github.com/opensearch-project/ml-commons.git, https://github.com/opensearch-project/neural-search.git, https://github.com/opensearch-project/security-analytics.git, https://github.com/opensearch-project/performance-analyzer.git]
Compatible components: [https://github.com/opensearch-project/performance-analyzer-rca.git, https://github.com/opensearch-project/opensearch-oci-object-storage.git]

BUILD SUCCESSFUL in 22m 15s

github-actions[bot] commented 9 months ago

Gradle Check (Jenkins) Run Completed with:

RESULT: FAILURE :x:
URL: https://build.ci.opensearch.org/job/gradle-check/22422/
CommitID: 1c5c0e0ae0718ccdf9d0d8e3657f07e34dc6a4a6 Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

BhumikaSaini-Amazon commented 9 months ago

Splitting this PR into incremental PRs. Will keep this PR to be merged last, with only the changes for _nodes/stats/ API integration.

opensearch-trigger-bot[bot] commented 9 months ago

Compatibility status:


> Task :checkCompatibility
Incompatible components: [https://github.com/opensearch-project/geospatial.git, https://github.com/opensearch-project/security.git, https://github.com/opensearch-project/notifications.git, https://github.com/opensearch-project/neural-search.git, https://github.com/opensearch-project/sql.git, https://github.com/opensearch-project/index-management.git, https://github.com/opensearch-project/security-analytics.git, https://github.com/opensearch-project/job-scheduler.git, https://github.com/opensearch-project/observability.git, https://github.com/opensearch-project/k-nn.git, https://github.com/opensearch-project/alerting.git, https://github.com/opensearch-project/cross-cluster-replication.git, https://github.com/opensearch-project/anomaly-detection.git, https://github.com/opensearch-project/ml-commons.git, https://github.com/opensearch-project/asynchronous-search.git, https://github.com/opensearch-project/common-utils.git, https://github.com/opensearch-project/performance-analyzer.git, https://github.com/opensearch-project/reporting.git]
Compatible components: [https://github.com/opensearch-project/opensearch-oci-object-storage.git, https://github.com/opensearch-project/performance-analyzer-rca.git]

BUILD SUCCESSFUL in 32m 39s

github-actions[bot] commented 9 months ago

Gradle Check (Jenkins) Run Completed with:

RESULT: FAILURE :x:
URL: https://build.ci.opensearch.org/job/gradle-check/22476/
CommitID: 65c21106a8b5238a30143bbd9231bbf8dbe29b64 Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

github-actions[bot] commented 9 months ago

Gradle Check (Jenkins) Run Completed with:

RESULT: FAILURE :x:
URL: https://build.ci.opensearch.org/job/gradle-check/24073/
CommitID: 192f35df78d08925be3f2ba375350c8e092bbb16 Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

github-actions[bot] commented 9 months ago

Compatibility status:

Checks if related components are compatible with change 192f35d

Incompatible components

Incompatible components: [https://github.com/opensearch-project/cross-cluster-replication.git]

Skipped components

Compatible components

github-actions[bot] commented 9 months ago

Compatibility status:

Checks if related components are compatible with change d90db16

Incompatible components

Incompatible components: [https://github.com/opensearch-project/cross-cluster-replication.git]

Skipped components

Compatible components

github-actions[bot] commented 9 months ago

Gradle Check (Jenkins) Run Completed with:

RESULT: FAILURE :x:
URL: https://build.ci.opensearch.org/job/gradle-check/24074/
CommitID: d90db1675e8433fc0abfd0b8824f71add07721c6 Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

github-actions[bot] commented 9 months ago

Compatibility status:

Checks if related components are compatible with change 0d45d99

Incompatible components

Skipped components

Compatible components

github-actions[bot] commented 9 months ago

Gradle Check (Jenkins) Run Completed with:

RESULT: FAILURE :x:
URL: https://build.ci.opensearch.org/job/gradle-check/24112/
CommitID: fa66abc9b213e8451b0f0b7d1160b43d478498cf Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

github-actions[bot] commented 9 months ago

Gradle Check (Jenkins) Run Completed with:

RESULT: FAILURE :x:
URL: https://build.ci.opensearch.org/job/gradle-check/24110/
CommitID: 0d45d9924875eabda153e38704bb4f9f18493578 Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

github-actions[bot] commented 9 months ago

Compatibility status:

Checks if related components are compatible with change fa66abc

Incompatible components

Skipped components

Compatible components

github-actions[bot] commented 9 months ago

Compatibility status:

Checks if related components are compatible with change 2c1a499

Incompatible components

Skipped components

Compatible components

github-actions[bot] commented 9 months ago

Gradle Check (Jenkins) Run Completed with:

RESULT: SUCCESS :white_check_mark:
URL: https://build.ci.opensearch.org/job/gradle-check/24117/
CommitID: 2c1a499cdb4b7bafe23beca17deeb353e2a3125c

codecov[bot] commented 9 months ago

Codecov Report

Merging #8908 (98d86ec) into main (1b41fa0) will decrease coverage by 0.06%. The diff coverage is 71.07%.

@@             Coverage Diff              @@
##               main    #8908      +/-   ##
============================================
- Coverage     71.20%   71.14%   -0.06%     
+ Complexity    58116    58066      -50     
============================================
  Files          4829     4830       +1     
  Lines        273740   273844     +104     
  Branches      39887    39895       +8     
============================================
- Hits         194912   194834      -78     
- Misses        62548    62641      +93     
- Partials      16280    16369      +89

Files Changed	Coverage Δ
...in/cluster/remotestore/stats/RemoteStoreStats.java	`96.24% <0.00%> (ø)`
...rg/opensearch/index/remote/RemoteSegmentStats.java	`86.15% <0.00%> (-12.10%)`	:arrow_down:
...opensearch/index/translog/RemoteTranslogStats.java	`82.08% <82.08%> (ø)`
...a/org/opensearch/index/translog/TranslogStats.java	`90.27% <83.87%> (-4.27%)`	:arrow_down:
...in/java/org/opensearch/index/shard/IndexShard.java	`69.36% <100.00%> (+0.01%)`	:arrow_up:

... and 487 files with indirect coverage changes

github-actions[bot] commented 9 months ago

Compatibility status:

Checks if related components are compatible with change b49c2a6

Incompatible components

Skipped components

Compatible components

github-actions[bot] commented 9 months ago

Gradle Check (Jenkins) Run Completed with:

RESULT: FAILURE :x:
URL: https://build.ci.opensearch.org/job/gradle-check/24278/
CommitID: b49c2a6ea739e9640e3c02948ae6c11504c05c2d Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

github-actions[bot] commented 9 months ago

Gradle Check (Jenkins) Run Completed with:

RESULT: FAILURE :x:
URL: https://build.ci.opensearch.org/job/gradle-check/24279/
CommitID: ea9d86e577eeae71a4ec77d0a28952f829cdadf9 Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

github-actions[bot] commented 9 months ago

Compatibility status:

Checks if related components are compatible with change ea9d86e

Incompatible components

Skipped components

Compatible components

github-actions[bot] commented 9 months ago

Gradle Check (Jenkins) Run Completed with:

RESULT: FAILURE :x:
URL: https://build.ci.opensearch.org/job/gradle-check/24340/
CommitID: 7837a802c9668841fb2c44e044e5e1cc080f23bf Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

github-actions[bot] commented 9 months ago

Compatibility status:

Checks if related components are compatible with change 7837a80

Incompatible components

Skipped components

Compatible components

github-actions[bot] commented 9 months ago

Gradle Check (Jenkins) Run Completed with:

RESULT: FAILURE :x:
URL: https://build.ci.opensearch.org/job/gradle-check/24343/
CommitID: ac944f938fdfbd9cad20823626d68a6ac7cdbeb8 Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

github-actions[bot] commented 9 months ago

Compatibility status:

Checks if related components are compatible with change ac944f9

Incompatible components

Skipped components

Compatible components

github-actions[bot] commented 9 months ago

Gradle Check (Jenkins) Run Completed with:

RESULT: FAILURE :x:
URL: https://build.ci.opensearch.org/job/gradle-check/24364/
CommitID: d97e18e9c6320464a27c48237f9f8a96d693199f Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

github-actions[bot] commented 9 months ago

Compatibility status:

Checks if related components are compatible with change d97e18e

Incompatible components

Incompatible components: [https://github.com/opensearch-project/neural-search.git, https://github.com/opensearch-project/sql.git, https://github.com/opensearch-project/anomaly-detection.git]

Skipped components

Compatible components

github-actions[bot] commented 9 months ago

Compatibility status:

Checks if related components are compatible with change 50a493c

Incompatible components

Incompatible components: [https://github.com/opensearch-project/neural-search.git, https://github.com/opensearch-project/sql.git, https://github.com/opensearch-project/anomaly-detection.git]

Skipped components

Compatible components

github-actions[bot] commented 9 months ago

Gradle Check (Jenkins) Run Completed with:

RESULT: FAILURE :x:
URL: https://build.ci.opensearch.org/job/gradle-check/24365/
CommitID: 50a493c67fab8834d1fb84cf867c025d6eef725a Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

github-actions[bot] commented 9 months ago

Compatibility status:

Checks if related components are compatible with change f18454e

Incompatible components

Incompatible components: [https://github.com/opensearch-project/neural-search.git, https://github.com/opensearch-project/sql.git, https://github.com/opensearch-project/anomaly-detection.git]

Skipped components

Compatible components

github-actions[bot] commented 9 months ago

Gradle Check (Jenkins) Run Completed with:

RESULT: SUCCESS :white_check_mark:
URL: https://build.ci.opensearch.org/job/gradle-check/24371/
CommitID: f18454ed4bab3c41e836aa8b25ab6a615d0d7ed7

github-actions[bot] commented 9 months ago

Compatibility status:

Checks if related components are compatible with change 55c7668

Incompatible components

Incompatible components: [https://github.com/opensearch-project/neural-search.git, https://github.com/opensearch-project/sql.git, https://github.com/opensearch-project/anomaly-detection.git]

Skipped components

Compatible components

github-actions[bot] commented 9 months ago

Gradle Check (Jenkins) Run Completed with:

RESULT: FAILURE :x:
URL: https://build.ci.opensearch.org/job/gradle-check/24402/
CommitID: 55c76687c441c0c6a4ccebb5572525a2c89f74ba Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

github-actions[bot] commented 9 months ago

Compatibility status:

Checks if related components are compatible with change c3ec8cb

Incompatible components

Skipped components

Compatible components

github-actions[bot] commented 9 months ago

Gradle Check (Jenkins) Run Completed with:

RESULT: FAILURE :x:
URL: https://build.ci.opensearch.org/job/gradle-check/24407/
CommitID: c3ec8cb1118c1180fab3d769deccd83d8daad4c3 Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

github-actions[bot] commented 9 months ago

Compatibility status:

Checks if related components are compatible with change f928522

Incompatible components

Incompatible components: [https://github.com/opensearch-project/sql.git, https://github.com/opensearch-project/anomaly-detection.git, https://github.com/opensearch-project/neural-search.git]

Skipped components

Compatible components

github-actions[bot] commented 9 months ago

Gradle Check (Jenkins) Run Completed with:

RESULT: FAILURE :x:
URL: https://build.ci.opensearch.org/job/gradle-check/24419/
CommitID: f928522d1bb465e173f07214b86edf0e853f982c Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

github-actions[bot] commented 9 months ago

Gradle Check (Jenkins) Run Completed with:

RESULT: FAILURE :x:
URL: https://build.ci.opensearch.org/job/gradle-check/24431/
CommitID: ed9067719f6e352d3eb9de0243efd4c1ddc8601a Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

github-actions[bot] commented 9 months ago

Compatibility status:

Checks if related components are compatible with change ed90677

Incompatible components

Incompatible components: [https://github.com/opensearch-project/anomaly-detection.git, https://github.com/opensearch-project/sql.git, https://github.com/opensearch-project/neural-search.git]

Skipped components

Compatible components

opensearch-project / OpenSearch

[Remote Store] Add support for Remote Translog Store upload stats in _nodes/stats/ API #8908

Description

Related Issues

8311

Check List

Gradle Check (Jenkins) Run Completed with:

Gradle Check (Jenkins) Run Completed with:

Gradle Check (Jenkins) Run Completed with:

Gradle Check (Jenkins) Run Completed with:

Gradle Check (Jenkins) Run Completed with:

Gradle Check (Jenkins) Run Completed with:

Gradle Check (Jenkins) Run Completed with:

Gradle Check (Jenkins) Run Completed with:

Gradle Check (Jenkins) Run Completed with:

Gradle Check (Jenkins) Run Completed with:

Gradle Check (Jenkins) Run Completed with:

Compatibility status:

Incompatible components

Skipped components

Compatible components

Compatibility status:

Incompatible components

Skipped components

Compatible components

Gradle Check (Jenkins) Run Completed with:

Compatibility status:

Incompatible components

Skipped components

Compatible components

Gradle Check (Jenkins) Run Completed with:

Gradle Check (Jenkins) Run Completed with:

Compatibility status:

Incompatible components

Skipped components

Compatible components

Compatibility status:

Incompatible components

Skipped components

Compatible components

Gradle Check (Jenkins) Run Completed with:

Codecov Report

Compatibility status:

Incompatible components

Skipped components

Compatible components

Gradle Check (Jenkins) Run Completed with:

Gradle Check (Jenkins) Run Completed with:

Compatibility status:

Incompatible components

Skipped components

Compatible components

Gradle Check (Jenkins) Run Completed with:

Compatibility status:

Incompatible components

Skipped components

Compatible components

Gradle Check (Jenkins) Run Completed with:

Compatibility status:

Incompatible components

Skipped components

Compatible components

Gradle Check (Jenkins) Run Completed with:

Compatibility status:

Incompatible components

Skipped components

Compatible components

Compatibility status:

Incompatible components

Skipped components

Compatible components

Gradle Check (Jenkins) Run Completed with:

Compatibility status:

Incompatible components

Skipped components

Compatible components

Gradle Check (Jenkins) Run Completed with:

Compatibility status:

Incompatible components

Skipped components