RE: No cert metrics when target has expired or invalid certificate #37

[wHiteeeeeeeee]

Hi, I tried to add:

modules:
tcp_insecure:
prober: tcp
tls_config:
insecure_skip_verify: true

to my ssl_exporter.yml, but i doesn't work. My ssl_exporter.yml file:

modules:
https_insecure:
prober: https
tls_config:
insecure_skip_verify: true
tcp:
prober: tcp
tcp_insecure:
prober: tcp
tls_config:
insecure_skip_verify: true

I need to add something more to yml to work perfectly? ssl_cert_not_after is working fine like other's metrics, but I don't have certs that expired in ssl_cert_not_after.

Thanks for your help!

[ribbybibby]

What does your scrape config in Prometheus look like? Are you setting the module parameter to tcp_insecure?

[wHiteeeeeeeee]

global:
  scrape_interval:     15s
  evaluation_interval: 15s
rule_files:
  - alert.rules.yml

alerting:
  alertmanagers:
  - static_configs:
    - targets:
      - 10.16.155.151:9093
scrape_configs:
  - job_name: prometheus
    static_configs:
      - targets: ['localhost:9090']
  - job_name: 'blackbox'
    metrics_path: /probe
    scheme: http
    params:
      module: [http_2xx]
    file_sd_configs:
          - files:
                - /etc/blackbox/targets.yml
    relabel_configs:
      - source_labels: [__address__]
        target_label: __param_target
      - source_labels: [__param_target]
        target_label: instance
      - target_label: __address__
        replacement: 10.16.155.151:9115  # The blackbox exporter's real hostname:port.
  - job_name: 'ssl'
    metrics_path: /probe
    scheme: http
    static_configs:
     - targets:
        - 'youtube.com:443'
        - 'prometheus.io:443'
        - 'cuk.pl:443'
        - 'launcher.uat.agent.nau.pl:443'
        - 'expired.badssl.com:443'
    relabel_configs:
      - source_labels: [__address__]
        target_label: __param_target
      - source_labels: [__param_target]
        target_label: instance
      - target_label: __address__
        replacement: 10.16.155.151:9219  # SSL exporter.
    tls_config:
      insecure_skip_verify: true

Do i need to add module tcp? Sorry I'm begginer in prometheus ;-)

[ribbybibby]

You should set the module in the params of the ssl scrape job:

  - job_name: 'ssl'
    metrics_path: /probe
    scheme: http
    params:
      module: ["tcp_insecure"] # <----- set the module in the params
    static_configs:
     - targets:
        - 'youtube.com:443'
        - 'prometheus.io:443'
        - 'cuk.pl:443'
        - 'launcher.uat.agent.nau.pl:443'
        - 'expired.badssl.com:443'
    relabel_configs:
      - source_labels: [__address__]
        target_label: __param_target
      - source_labels: [__param_target]
        target_label: instance
      - target_label: __address__
        replacement: 10.16.155.151:9219  # SSL exporter.

[wHiteeeeeeeee]

Now i got something like this:

global:
  scrape_interval:     15s
  evaluation_interval: 15s
rule_files:
  - alert.rules.yml

alerting:
  alertmanagers:
  - static_configs:
    - targets:
      - 10.16.155.151:9093
scrape_configs:
  - job_name: prometheus
    static_configs:
      - targets: ['localhost:9090']
  - job_name: 'blackbox'
    metrics_path: /probe
    scheme: http
    params:
      module: [http_2xx]
    file_sd_configs:
          - files:
                - /etc/blackbox/targets.yml
    relabel_configs:
      - source_labels: [__address__]
        target_label: __param_target
      - source_labels: [__param_target]
        target_label: instance
      - target_label: __address__
        replacement: 10.16.155.151:9115  # The blackbox exporter's real hostname:port.
  - job_name: 'ssl'
    metrics_path: /probe
    scheme: http
    params:
      module: ["tcp_insecure"]
    static_configs:
     - targets:
        - 'youtube.com'
        - 'prometheus.io:443'
        - 'cuk.pl:443'
        - 'launcher.uat.agent.nau.pl:443'
        - 'http://expired.badssl.com'
        - 'http://wrong.host.badssl.com'
    relabel_configs:
      - source_labels: [__address__]
        target_label: __param_target
      - source_labels: [__param_target]
        target_label: instance
      - target_label: __address__
        replacement: 10.16.155.151:9219  # SSL exporter.

and ssl_exporter.yml:

modules:
  https:
    prober: https
  http:
    prober: http
    tls_config:
      insecure_skip_verify: true
  tcp_insecure:
    prober: tcp
    tls_config:
      insecure_skip_verify: true

and i doesn't work neither. What is wrong?

[ribbybibby]

Are there any logs from the ssl_exporter process?

[wHiteeeeeeeee]

I see there is some problem with yml file. Maybe any suggestions what can be wrong with that?

[ribbybibby]

Ah, looks like you're running 2.1.1 which doesn't support the timeout parameter in modules. Upgrade to 2.2.0 for that.

[wHiteeeeeeeee]

Ok I'll try ;)

[wHiteeeeeeeee]

It seems like nothing changed :/ I tried to change yml file from ssl_exporter to that:

modules:
  https:
    prober: https
  http:
    prober: http
  tcp_insecure:
    prober: tcp
    tls_config:
      insecure_skip_verify: true

and still the same :/

[ribbybibby]

error reading config file: open /etc/ssl_exporter/ssl_exporter.yml: no such file or directory

[wHiteeeeeeeee]

Yea I see, but I don't know what seems to be wrong with that file. ssl_exporter is running but in journalctl pops error. Maybe any suggestions what it may look like simple exemplary of yml file? I don't have kubertenes, I just downloaded package from Your releases and only working on ssl_exporter yml and prometheus.yml.

[ribbybibby]

The logs suggest that the file isn't there at the path /etc/ssl_exporter/ssl_exporter.yml - is it? Or perhaps there's some issue with the permissions of that file?

[wHiteeeeeeeee]

Path is good, permissions seems to be good too:

-rwxr-xr-x   1 ssl_exporter ssl_exporter   142 Dec 11 13:14 ssl_exporter.yml

If i change something in ssl yml, for example module, and i attach to prometheus.yml, it's working.

[ribbybibby]

It's difficult for me to debug this without more visibility into your environment. What does your systemd service file look like? How are you running the exporter, with a binary or as a container?

[wHiteeeeeeeee]

[Unit]
Description=SSL Exporter Service
Wants=network-online.target
After=network-online.target

[Service]
Type=simple
User=ssl_exporter
Group=ssl_exporter
ExecStart=/usr/local/bin/ssl_exporter \
  --config.file=/etc/ssl_exporter/ssl_exporter.yml \
  --web.listen-address="10.16.155.151:9219" \

Restart=always

[Install]
WantedBy=multi-user.target

I created ssl_exporter.service and it seems to work, because it's active, but it something seems to be wrong with ssl_exporter.yml, like some module or something is missing.

[ribbybibby]

Okay, let's put the no such file or directory errors to one side and say that your service is running.

If you are getting ssl_tls_connect 0 (this is now ssl_probe_success in the latest version) for some targets then there should be some logs emitted by the ssl_exporter that explain why the probe failed.

Something like:

ERRO[0228] error=x509: certificate has expired or is not yet valid: current time 2020-12-12T13:13:49Z is after 2015-04-12T23:59:59Z target=expired.badssl.com:443 prober=tcp timeout=10s  source="ssl_exporter.go:93"

This will give you an indication of what the issue is.

Looking more closely at your screenshot and the latest scrape config I can see a lot of targets that are formatted badly:

        - 'youtube.com'
        - 'http://expired.badssl.com'
        - 'http://wrong.host.badssl.com'

These should all be in <host>:<port> format:

- 'youtube.com:443'
- 'expired.badssl.com:443'
- 'wrong.host.badssl.com:443'

[wHiteeeeeeeee]

Wow, now it works, i changed to host:port and it started to take all certs :D Thanks a lot for your help!