HELM: Trying to deploy on local DEV Openshift (no S3) fails

[danielmarom78]

Trying to do a simple install on my local DEV Openshift (running with CRC). I have no S3

Describe the bug

I'm getting crashloopback on multiple components with 'No s3 endpoint in config file'

To Reproduce

helm -n mimir install mimir grafana/mimir-distributed -f local.yaml

Where the local.yaml has only

mimir:
  structuredConfig:
    common:
      storage:
        backend: filesystem

minio:
  enabled: false

rbac:
  create: false
  type: scc
  podSecurityContext:
    fsGroup: null
    runAsGroup: null
    runAsUser: null
rollout_operator:
  podSecurityContext:
    fsGroup: null
    runAsGroup: null
    runAsUser: null

Expected behavior

Successful deployment

Environment

Openshift 4.13.6 local env VM running on CRC

Additional Context

[dimitarvdimitrov]

the values of the per-section s3 config blocks take precedence over the common block. You can override alertmanager_storage, blocks_storage and ruler_storage separately.

I think it would be a good idea to change the default values to use the common block to avoid this problem.

[dimitarvdimitrov]

i saw that with minio.enable: false only the blocks_storage section gets backend: s3 explicitly. If we make this conditional on minio.enabled, then this problem shouldn't occur. Can you check whether setting blocks_store.backend: filesystem in addition to common.storage.backend fixes the problem?

[dionisiemoscalu]

hi @dimitarvdimitrov any resolution on it ? (i have same configuration and same output)

[echozio]

@dionisiemoscalu the workaround is to set blocks_store.backend explicitly in addition to common.storage.backend. It doesn't fall back to common.storage.backend because it's always set explicitly to to s3 in config template.

I can't imagine it's meant to work like this, so I'll open a PR to change it.