cortexproject / cortex

A horizontally scalable, highly available, multi-tenant, long term Prometheus.
https://cortexmetrics.io/
Apache License 2.0
5.48k stars 801 forks source link

Azure storage needs the endpoint suffix after v1.15.0 #5449

Open friedrichg opened 1 year ago

friedrichg commented 1 year ago

Describe the bug After https://github.com/cortexproject/cortex/commit/da2ad99f2c258537701c2b427195029c838612df all the containers that depend on block storage resources hang on authorization failing to start. There is no message.

it's necessary to specify the endpoint suffix for all values, In my case I had to specify blob.core.windows.net the empty value is no longer valid.

  # Azure storage endpoint suffix without schema. The account name will be
  # prefixed to this value to create the FQDN
  # CLI flag: -ruler-storage.azure.endpoint-suffix
  [endpoint_suffix: <string> | default = ""]

Expected behavior Cortex should now fail to start if the suffix value is not provided

Additional Context

One can see the problem with

$ cat alertmanager.yaml
alertmanager:
  data_dir: /data
  enable_api: true
alertmanager_storage:
  azure:
    account_key: dmFsaWRhY2NvdW50a2V5 # validstoragekey encoded with base64
    account_name: validstorageaccountname
    container_name: validcontainername
    # endpoint_suffix: blob.core.windows.net
  backend: azure
server:
  log_level: debug
target: alertmanager

$ docker run -e AZURE_SDK_GO_LOGGING=all -v $PWD/alertmanager.yaml:/alertmanager.yaml cortexproject/cortex:master-da2ad99 -config.file=/alertmanager.yaml
...
Get "https://validstorageaccountname./validcontainername?restype=container": dial tcp: lookup validstorageaccountname. on 192.168.5.3:53: no such host
...
alanprot commented 1 year ago

Seems related to https://github.com/thanos-io/thanos/issues/6175

moki1202 commented 1 year ago

@friedrichg @alanprot here's my approach to this, I can add a validate function in pkg/storage/bucket/azure/config.go which checks whether endpoint_suffix is empty or not, and execute the function right after parsing the command line flags. If error occurs then I can just do os.Exit(1). Is this approach correct?

friedrichg commented 1 year ago

@moki1202 fixing in thanos should be more appropriate

yeya24 commented 2 months ago

Do we still need a fix for this?