[tempo-distributed] access-key-id you provided does not exist in our records

[alita1991]

I cannot successfully deploy the helm chart for tempo-distrubuted using the values from the example. I tested without success 1.6.3, 1.6.4, and 1.6.6 versions. I even tested with an external minio that I used when I configured mimir and loki, and I got the same issue.

Is there anything I missed that I should do to ensure that Minio is used as a backend store?

Config used

---
global:
  clusterDomain: 'cluster.local'
gateway:
  enabled: true
minio:
  enabled: true
storage:
  trace:
    backend: s3
    s3:
      access_key: 'grafana-tempo'
      secret_key: 'supersecret'
      bucket: 'tempo-traces'
      endpoint: 'tempo-minio:9000'
      insecure: true
traces:
  otlp:
    http:
      enabled: true
    grpc:
      enabled: true
distributor:
  config:
    log_received_spans:
      enabled: true

Error log level=error ts=2023-10-02T11:48:42.53082433Z caller=main.go:111 msg="error running Tempo" err="failed to init module services error initialising module: store: failed to create store unexpected error from ListObjects on tempo-traces: The Access Key Id you provided does not exist in our records."

[Sheikh-Abubaker]

Hi @alita1991 checkout if this might help with Tempo Helm chart values

[omri-shilton]

im also experiencing this @alita1991 did you end up solving this?

[marioanton]

Likely due to usage report being enabled. That will send your statistics to grafana labs.

tempo:
  reportingEnabled: false

did sort it out for me.

[devgig]

Same for me.