grafana / helm-charts

Apache License 2.0
1.66k stars 2.27k forks source link

[loki] aws insecure parameter is not effective #1025

Open scybZhangliang opened 2 years ago

scybZhangliang commented 2 years ago

I've set the insecure parameter was true , but there is still output the "https://minio.default.svc.cluster.local/loki-data/fake/85a8507d6831e392%3A17ede5a9024%3A17ede5f2fae%3A4684e217\": x509: certificate signed by unknown authority error messge in the log. below is my aws config snippet:

  storage_config:
    boltdb_shipper:
      active_index_directory: /data/loki/boltdb-shipper-active
      cache_location: /data/loki/boltdb-shipper-cache
      cache_ttl: 24h         # Can be increased for faster performance over longer query periods, uses more disk space
      shared_store: s3
    filesystem:
      directory: /data/loki/chunks
    aws:
      endpoint: https://minio.default.svc.cluster.local
      insecure: true
      bucketnames: loki-data
      access_key_id: minioadmin
      secret_access_key: minioadmin
      s3forcepathstyle: true

need your help, thks!

gerardjp commented 2 years ago

I'm seeing the same issue, except I'm on a local minio S3 install and have the url configured as aws:s3 instead of aws:endpoint using a valid wildcard certificate. This setup works with self-signed certs behind the minio install, but loki doesn't seem handle (or validate) the CA chain somehow.

UPDATE: Figured it out. My Minio backend needs to serve (and thus have) the CA certs bundled with the actual cert having the domainname.

Testeable with:

openssl s_client -showcerts -servername s3.blabla.nl -connect s3.blabla.nl:9000

Output displayed gave:

Verify return code: 21 (unable to verify the first certificate)

tman5 commented 1 year ago

I have this same issue where insecure: true does nothing. I do not have the issue with my cert chain though. The chain has all the certs and no errors using openssl

Forestsoft-de commented 1 year ago

Did you tried:

storage_config:
    boltdb_shipper:
      active_index_directory: /data/loki/boltdb-shipper-active
      cache_location: /data/loki/boltdb-shipper-cache
      cache_ttl: 24h         # Can be increased for faster performance over longer query periods, uses more disk space
      shared_store: s3
    filesystem:
      directory: /data/loki/chunks
    aws:
      endpoint: https://minio.default.svc.cluster.local
      insecure: true
      bucketnames: loki-data
      access_key_id: minioadmin
      secret_access_key: minioadmin
      s3forcepathstyle: true
      http_config:
            insecure_skip_verify: true

br Sebastian

tman5 commented 1 year ago

Thanks. That seems to have cleared the error. Still getting a failure in Grafana when adding the data source and I haven't found any errors in the stack yet.