grafana / loki

Like Prometheus, but for logs.
https://grafana.com/loki
GNU Affero General Public License v3.0
23.8k stars 3.43k forks source link

Loki is not pushing logs to s3 and no errors are shown in container logs #13754

Open AB-291 opened 2 months ago

AB-291 commented 2 months ago

I am trying to setup loki locally by pulling docker image and connecting loki with s3 for logs storage. I have this loki config yaml file.

auth_enabled: false

server:
  http_listen_port: 3100
  grpc_listen_port: 9095

ingester:
  lifecycler:
    address: 127.0.0.1
    ring:
      kvstore:
        store: inmemory
      replication_factor: 1
  chunk_idle_period: 5m
  chunk_retain_period: 30s
  max_transfer_retries: 0
  chunk_target_size: 1536000
  chunk_block_size: 262144

schema_config:
  configs:
    - from: 2023-01-01
      store: tsdb
      object_store: aws
      schema: v13
      index:
        prefix: index_
        period: 24h

storage_config:
  tsdb_shipper:
    active_index_directory: /loki/index
    cache_location: /loki/index_cache
    cache_ttl: 24h
  aws:
    s3: s3://AKIAU5DVIDKCSUWJSOLM:2eJoL28TUmdlqCsmM4ZPpmnyvuJVL@ap-south-1
    bucketnames: sparrow-logs

compactor:
  working_directory: /loki/compactor
  shared_store: s3

limits_config:
  enforce_metric_name: false
  reject_old_samples: true
  reject_old_samples_max_age: 168h

chunk_store_config:
  max_look_back_period: 0s

table_manager:
  retention_deletes_enabled: true
  retention_period: 168h

ruler:
  storage:
    type: local
    local:
      directory: /loki/rules
  rule_path: /loki/rules
  ring:
    kvstore:
      store: inmemory
  enable_api: true

It doesn't show any error. In fact it shows some logs like this

2024-08-05 09:02:54 level=info ts=2024-08-05T04:02:54.157209799Z caller=table.go:318 msg="handing over indexes to shipper index_19940"
2024-08-05 09:02:54 level=info ts=2024-08-05T04:02:54.157223507Z caller=table.go:334 msg="finished handing over table index_19940"
2024-08-05 09:02:54 level=info ts=2024-08-05T04:02:54.164623632Z caller=index_set.go:107 msg="finished uploading table index_19940"
2024-08-05 09:02:54 level=info ts=2024-08-05T04:02:54.164649799Z caller=index_set.go:185 msg="cleaning up unwanted indexes from table index_19940"
2024-08-05 09:37:16 level=info ts=2024-08-05T04:37:16.79647667Z caller=index_set.go:86 msg="uploading table index_19940"
2024-08-05 09:37:16 level=info ts=2024-08-05T04:37:16.796544212Z caller=index_set.go:107 msg="finished uploading table index_19940"
2024-08-05 09:37:16 level=info ts=2024-08-05T04:37:16.796556754Z caller=index_set.go:185 msg="cleaning up unwanted indexes from table index_19940"
2024-08-05 09:37:16 level=info ts=2024-08-05T04:37:16.796595795Z caller=table.go:318 msg="handing over indexes to shipper index_19940"
2024-08-05 09:37:16 level=info ts=2024-08-05T04:37:16.796608004Z caller=table.go:334 msg="finished handing over table index_19940"

But when I check my s3 bucket it remain empty. I have verified my s3 creds by uploading file from my terminal and they are correct.

JStickler commented 3 days ago

Configuration questions have a better chance of being answered if you ask them on the community forums.