opensearch-project / OpenSearch

🔎 Open source distributed and RESTful search engine.
https://opensearch.org/docs/latest/opensearch/index/
Apache License 2.0
9.84k stars 1.83k forks source link

Increase segrep pressure checkpoint default limit to 30 #16577

Closed gbbafna closed 1 week ago

gbbafna commented 2 weeks ago

Description

The default segrep.pressure.time.limit is 5 mins , while default segrep.pressure.checkpoint.limit is just 4 which only allows for 40 sec lag given the refresh interval of 10 secs . Though the default refresh interval is 1sec, for remote backed clusters, we advice >10 sec due to prohibitive costs and high resource consumption of segment upload every sec. This PR increases segrep.pressure.checkpoint.limit to 10 so that both checkpoint limit and time limit equates to same 300 sec and we don't give unnecessary rejections to the user .

This relaxation is due to the fact that we don't have multipart download or block level fetches in download part, hence the observed lag on clusters are on a higher side (100s of seconds) during force merges .

Check List

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license. For more information on following Developer Certificate of Origin and signing off your commits, please check here.

github-actions[bot] commented 2 weeks ago

:x: Gradle check result for 965595e92d4cde673c03f1a4e9d1dff82a0b9ef7: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

github-actions[bot] commented 2 weeks ago

:x: Gradle check result for b14c7c8efdfad9ff7761854d443ac0f11d805e88: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

github-actions[bot] commented 2 weeks ago

:white_check_mark: Gradle check result for 3fef8c0d78a6e8c5beb828922b50ce800bd4909e: SUCCESS

codecov[bot] commented 2 weeks ago

Codecov Report

All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 72.07%. Comparing base (e688388) to head (b2ccfeb). Report is 1 commits behind head on main.

Additional details and impacted files ```diff @@ Coverage Diff @@ ## main #16577 +/- ## ============================================ - Coverage 72.15% 72.07% -0.08% + Complexity 65128 65084 -44 ============================================ Files 5315 5315 Lines 303573 303573 Branches 43925 43925 ============================================ - Hits 219036 218807 -229 - Misses 66565 66773 +208 - Partials 17972 17993 +21 ```

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

github-actions[bot] commented 1 week ago

:white_check_mark: Gradle check result for 67bdeafab46ee09769096c1b334ed3dcff4b6fd3: SUCCESS

github-actions[bot] commented 1 week ago

:white_check_mark: Gradle check result for b2ccfeba1259d0b881aa3ea91c79217ddc31f3df: SUCCESS