dockstore / dockstore

Our VM/Docker sharing infrastructure and management component
https://dockstore.org/
Apache License 2.0
116 stars 27 forks source link

SEAB-6388: No longer periodically log diagnostic database pool information #5882

Closed svonworl closed 1 month ago

svonworl commented 1 month ago

Description This PR changes the webservice to no longer periodically log information about the database pool. As of https://github.com/dockstore/dockstore/pull/5873, we now submit database pool information as CloudWatch metrics. I left the corresponding logging method in the helper class, so we can call it from other code if we need to. For example, given some particular exceptional condition, we might invoke it from adjacent code to give us more details, or similar.

We still periodically log:

As part of this ticket, I thought about whether we could summarize either of the above as metrics and eliminate the logging.

Regarding the filesystem information, the logical summary metric is "how much file storage is available/consumed?", which we'll probably add as part of https://ucsc-cgl.atlassian.net/browse/SEAB-4906. However, the df output could very well contain subtle clues that are diagnostically valuable.

It's more difficult to figure out how to summarize the memory pool details. Within, there's lots of different information that could be useful in various situations.

So, for now, we'll continue to periodically log filesystem and memory pool info. The details of both could help us to reason about previously-unencountered forms of space and memory problems.

Review Instructions Check the logs, and confirm that database pool information is no longer periodically logged.

Issue https://ucsc-cgl.atlassian.net/browse/SEAB-6388

Security and Privacy

No concerns.

e.g. Does this change...

Please make sure that you've checked the following before submitting your pull request. Thanks!

codecov[bot] commented 1 month ago

Codecov Report

All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 73.89%. Comparing base (e6d0782) to head (244c9c9).

Additional details and impacted files ```diff @@ Coverage Diff @@ ## develop #5882 +/- ## ========================================== Coverage 73.89% 73.89% Complexity 5264 5264 ========================================== Files 371 371 Lines 19190 19189 -1 Branches 2006 2006 ========================================== Hits 14180 14180 + Misses 4053 4052 -1 Partials 957 957 ``` | [Flag](https://app.codecov.io/gh/dockstore/dockstore/pull/5882/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=dockstore) | Coverage Δ | | |---|---|---| | [bitbuckettests](https://app.codecov.io/gh/dockstore/dockstore/pull/5882/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=dockstore) | `27.14% <ø> (+<0.01%)` | :arrow_up: | | [integrationtests](https://app.codecov.io/gh/dockstore/dockstore/pull/5882/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=dockstore) | `58.49% <ø> (+<0.01%)` | :arrow_up: | | [languageparsingtests](https://app.codecov.io/gh/dockstore/dockstore/pull/5882/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=dockstore) | `11.07% <ø> (+<0.01%)` | :arrow_up: | | [localstacktests](https://app.codecov.io/gh/dockstore/dockstore/pull/5882/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=dockstore) | `21.69% <ø> (+<0.01%)` | :arrow_up: | | [toolintegrationtests](https://app.codecov.io/gh/dockstore/dockstore/pull/5882/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=dockstore) | `30.50% <ø> (+<0.01%)` | :arrow_up: | | [unit-tests_and_non-confidential-tests](https://app.codecov.io/gh/dockstore/dockstore/pull/5882/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=dockstore) | `28.47% <ø> (+<0.01%)` | :arrow_up: | | [workflowintegrationtests](https://app.codecov.io/gh/dockstore/dockstore/pull/5882/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=dockstore) | `38.68% <ø> (+<0.01%)` | :arrow_up: | Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=dockstore#carryforward-flags-in-the-pull-request-comment) to find out more.

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.