search

kubermatic / mla

MLA (Monitoring, logging, alerting) solution for KKP.
Apache License 2.0
8 stars 10 forks source link

Fixes for issues observed in monitoring medium to large user-clusters #77

Closed dharapvj closed 2 years ago

dharapvj commented 2 years ago

Fixes #76

This PR brings following fixes:

  1. If we have too many files in Minio - minio pod cannot mount the volume. (more details of the issue and fix suggested in #76)
  2. Cortex Compactor fails to start if we have too many metrics in the storage. Cortex team has provided this suggestion to turn off the deleted_blocks_mark migration.
  3. We observed that some of pods were not getting scraped due to random limit of 30 labels defaulted in cortex chart. So relaxed this limit a bit to 40.
kubermatic-bot commented 2 years ago

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: dharapvj To complete the pull request process, please assign rastislavs after the PR has been reviewed. You can assign the PR to them by writing /assign @rastislavs in a comment when ready.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files: - **[OWNERS](https://github.com/kubermatic/mla/blob/main/OWNERS)** Approvers can indicate their approval by writing `/approve` in a comment Approvers can cancel approval by writing `/approve cancel` in a comment
dharapvj commented 2 years ago

Closing in favor of #81