prometheus / blackbox_exporter

Blackbox prober exporter
https://prometheus.io
Apache License 2.0
4.62k stars 1.05k forks source link

Custom CA cert #959

Open downright-ux opened 2 years ago

downright-ux commented 2 years ago

I 'm trying to configure blackbox to work with our custom CA . I see issue in the logs .

alertmanager       | ts=2022-08-17T17:41:34.355Z caller=notify.go:732 level=warn component=dispatcher receiver=team-1 integration=email[0] msg="Notify attempt failed, will retry later" attempts=1 err="send STARTTLS command: x509: certificate signed by unknown authority"

My configuration for blackbox.yaml

modules:
  http_2xx:
    prober: http
  http_post_2xx:
    prober: http
    http:
      method: POST
      tls_config:
        ca_file: "/opt/bitnami/blackbox-exporter/cacert.crt"
  http_custom_ca:
    prober: http
    http:
      method: GET
      tls_config:
        ca_file: "/opt/bitnami/blackbox-exporter/cacert.crt"  
  tcp_connect:
    prober: tcp
  pop3s_banner:
    prober: tcp
    tcp:
      query_response:
      - expect: "^+OK"
      tls: true
      tls_config:
        insecure_skip_verify: false
  grpc:
    prober: grpc
    grpc:
      tls: true
      preferred_ip_protocol: "ip4"
  grpc_plain:
    prober: grpc
    grpc:
      tls: false
      service: "service1"
  ssh_banner:
    prober: tcp
    tcp:
      query_response:
      - expect: "^SSH-2.0-"
      - send: "SSH-2.0-blackbox-ssh-check"
  irc_banner:
    prober: tcp
    tcp:
      query_response:
      - send: "NICK prober"
      - send: "USER prober prober prober :prober"
      - expect: "PING :([^ ]+)"
        send: "PONG ${1}"
      - expect: "^:[^ ]+ 001"
  icmp:
    prober: icmp
and docker-compose looks like this
```
version: '3'

services: blackbox-exporter: container_name: blackbox-exporter image: bitnami/blackbox-exporter network_mode: host restart: always ports:

therealcodlord commented 8 months ago

@downright-ux did you ever solve this? I have got the exact same issue.

My hosting provider did some migration and appears to have changed my Certificate Authority. Both wget and curl to my website was suddenly failing due to certificate errors. I solved that by manually downloading the CA certificate and updating certificates (sudo dpkg-reconfigure ca-certificates) on Ubuntu.

Blackbox probes started failing at the same time and I assume it's the same problem. I tried to specify the certificate file using tls_config and ca_file as you have done in your config above. But it makes no difference, still getting 403 forbidden errors with or without the tls_config.

I also tried adding insecure_skip_verify: true but no difference.

marinafrank commented 4 months ago

Try to place your custom CA certs in this location (excerpt from helm chart values)

extraSecretMounts:
  - name: custom-ca
    mountPath: /etc/ssl/certs/Custom-Issuing-CA.pem
    subPath: "Custom-Issuing-CA.pem"
    secretName: custom-ca
    readOnly: true
    defaultMode: 420