grafana / loki

Like Prometheus, but for logs.
https://grafana.com/loki
GNU Affero General Public License v3.0
23.91k stars 3.45k forks source link

Not able to store custom endpoint for S3 Chunk #11289

Open vadlakiran opened 11 months ago

vadlakiran commented 11 months ago

we have rook-ceph object storage and we are trying to install loki using the ceph object storage we have configured like below as s3.

storage: bucketNames: chunks: chunks ruler: chunks admin: chunks s3: accessKeyId: id given secretAccessKey: key given endpoint: http://rook-ceph-rgw-object-store.rook-ceph.svc.cluster.local/chunks insecure: true s3forcepathstyle: true type: s3

we are getting below issue, its appending the endpoint like below, can you please give me solution to use the rook-ceph object storage

` level=error ts=2023-11-22T09:18:26.158906343Z caller=flush.go:143 org_id=fake msg="failed to flush" err="failed to flush chunks: store put chunk: RequestError: send request failed\ncaused by: Put \"http://chunks.rook-ceph-rgw-object-store.rook-ceph.svc.cluster.local/fake/a1f567fb8196af22/18bf5d21296%3A18bf5d21297%3A1f8dbcc7\": dial tcp: lookup chunks.rook-ceph-rgw-object-store.rook-ceph.svc.cluster.local on 169.254.25.10:53: no such host, num_chunks: 1, labels: {app=\"loki\", component=\"backend\", container=\"loki-sc-rules\", filename=\"/var/log/pods/default_loki-backend-2_af6267be-4e5b-401c-9669-c58fe964684f/loki-sc-rules/0.log\", instance=\"loki\", job=\"default/loki\", namespace=\"default\", node_name=\"mongodb2\", pod=\"loki-backend-2\"}"

`

Describe the bug A clear and concise description of what the bug is.

To Reproduce Steps to reproduce the behavior:

  1. Started Loki (SHA or version)
  2. Started Promtail (SHA or version) to tail '...'
  3. Query: {} term

Expected behavior A clear and concise description of what you expected to happen.

Environment:

Screenshots, Promtail config, or terminal output If applicable, add any output to help explain your problem.

vadlakiran commented 9 months ago

@mizeng @ningyougang could you please help above issue with ceph with loki