opensearch-project / sql

Query your data using familiar SQL or intuitive Piped Processing Language (PPL)
https://opensearch.org/docs/latest/search-plugins/sql/index/
Apache License 2.0
121 stars 140 forks source link

Call LeaseManager for BatchQuery #3153

Closed ykmr1224 closed 1 week ago

ykmr1224 commented 1 week ago

Description

Related Issues

n/a

Check List

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license. For more information on following Developer Certificate of Origin and signing off your commits, please check here.

codecov[bot] commented 1 week ago

Codecov Report

All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 94.48%. Comparing base (cfe38d7) to head (dba09c4).

Additional details and impacted files ```diff @@ Coverage Diff @@ ## main #3153 +/- ## ========================================= Coverage 94.48% 94.48% - Complexity 5426 5428 +2 ========================================= Files 528 528 Lines 15452 15456 +4 Branches 1025 1025 ========================================= + Hits 14600 14604 +4 Misses 804 804 Partials 48 48 ``` | [Flag](https://app.codecov.io/gh/opensearch-project/sql/pull/3153/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=opensearch-project) | Coverage Δ | | |---|---|---| | [sql-engine](https://app.codecov.io/gh/opensearch-project/sql/pull/3153/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=opensearch-project) | `94.48% <100.00%> (+<0.01%)` | :arrow_up: | Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=opensearch-project#carryforward-flags-in-the-pull-request-comment) to find out more.

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

ykmr1224 commented 1 week ago

CI failure is due to coverage caused by: https://github.com/opensearch-project/sql/pull/3063

ykmr1224 commented 1 week ago
  1. Does the current class model meet the requirements for extending to another implementation?

https://github.com/opensearch-project/sql/blob/cfe38d7d6678dfbd5597f1098a1a47e29f84a708/async-query/src/main/java/org/opensearch/sql/spark/transport/config/AsyncExecutorServiceModule.java#L230-L232

Yes, DQS will internally check running jobs and limit new execution.

  1. Will this code change to BatchQueryHandler and RefreshQueryHandler affect the present logic of examining ConcurrentRefreshJobRule?

It won't affect. I excluded Batch query from ConcurrentRefreshJobRule.

ykmr1224 commented 1 week ago

Could you elaborate a little what was the issue?

DQS couldn't cover the Batch query to avoid too many concurrent execution.