scale on connection load - prototype

denis-yuen commented 2 months ago

Description This adds the reporting of database metrics to the healthcheck connection pool endpoint. Active, idle, size of the pool are reported along with calculated load (active connections over total size). The idea is to report this information to CloudWatch metrics so that we can develop auto-scaling policies (described in the linked ticket) using that data.

Note that the healthcheck endpoint is configured to be called by ECS every 60 seconds up from 30 seconds.

Follow-up ticket should investigate creating a custom reporter for dropwizatd metrics instead https://metrics.dropwizard.io/4.2.0/getting-started.html#other-reporting instead of hitchhiking on the healthcheck endpoint.

https://ucsc-cgl.atlassian.net/browse/SEAB-6401

Review Instructions TBD, will probably need benchmarking Use https://ucsc-cgl.atlassian.net/browse/SEAB-6402 for tracking

Issue https://ucsc-cgl.atlassian.net/browse/SEAB-5858

Security and Privacy

Metrics PUT temporarily triggerable via healthcheck endpoint

[ ] Security and Privacy assessed

Please make sure that you've checked the following before submitting your pull request. Thanks!

[x] Check that you pass the basic style checks and unit tests by running mvn clean install
[x] Ensure that the PR targets the correct branch. Check the milestone or fix version of the ticket.
[x] Follow the existing JPA patterns for queries, using named parameters, to avoid SQL injection
[x] If you are changing dependencies, check the Snyk status check or the dashboard to ensure you are not introducing new high/critical vulnerabilities
[x] Assume that inputs to the API can be malicious, and sanitize and/or check for Denial of Service type values, e.g., massive sizes
[x] Do not serve user-uploaded binary images through the Dockstore API
[x] Ensure that endpoints that only allow privileged access enforce that with the @RolesAllowed annotation
[x] Do not create cookies, although this may change in the future
[x] If this PR is for a user-facing feature, create and link a documentation ticket for this feature (usually in the same milestone as the linked issue). Style points if you create a documentation PR directly and link that instead.

codecov[bot] commented 2 months ago

Codecov Report

Attention: Patch coverage is 85.36585% with 6 lines in your changes are missing coverage. Please review.

Project coverage is 74.53%. Comparing base (1550b54) to head (e494629).

Files	Patch %	Lines
...ebservice/resources/ConnectionPoolHealthCheck.java	85.00%	6 Missing :warning:

Additional details and impacted files

```diff @@ Coverage Diff @@ ## develop #5872 +/- ## ============================================= + Coverage 71.45% 74.53% +3.07% - Complexity 5026 5245 +219 ============================================= Files 368 368 Lines 18945 18981 +36 Branches 1992 1992 ============================================= + Hits 13537 14147 +610 + Misses 4455 3883 -572 + Partials 953 951 -2 ``` | [Flag](https://app.codecov.io/gh/dockstore/dockstore/pull/5872/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=dockstore) | Coverage Δ | | |---|---|---| | [bitbuckettests](https://app.codecov.io/gh/dockstore/dockstore/pull/5872/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=dockstore) | `27.22% <21.95%> (-0.02%)` | :arrow_down: | | [integrationtests](https://app.codecov.io/gh/dockstore/dockstore/pull/5872/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=dockstore) | `58.97% <85.36%> (+0.06%)` | :arrow_up: | | [languageparsingtests](https://app.codecov.io/gh/dockstore/dockstore/pull/5872/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=dockstore) | `11.10% <21.95%> (+0.01%)` | :arrow_up: | | [localstacktests](https://app.codecov.io/gh/dockstore/dockstore/pull/5872/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=dockstore) | `21.71% <21.95%> (-0.01%)` | :arrow_down: | | [toolintegrationtests](https://app.codecov.io/gh/dockstore/dockstore/pull/5872/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=dockstore) | `30.59% <21.95%> (+0.60%)` | :arrow_up: | | [unit-tests_and_non-confidential-tests](https://app.codecov.io/gh/dockstore/dockstore/pull/5872/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=dockstore) | `28.56% <21.95%> (+5.13%)` | :arrow_up: | | [workflowintegrationtests](https://app.codecov.io/gh/dockstore/dockstore/pull/5872/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=dockstore) | `38.84% <21.95%> (+7.85%)` | :arrow_up: | Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=dockstore#carryforward-flags-in-the-pull-request-comment) to find out more.

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

sonarcloud[bot] commented 2 months ago

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
85.4% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarCloud

denis-yuen commented 1 month ago

Continued and replaced by https://github.com/dockstore/dockstore/pull/5873

dockstore / dockstore

scale on connection load - prototype #5872

Codecov Report

Quality Gate passed