probe success 0

[greenspartan]

Hi guys,

I have the error probe_success 0 when trying to perform following curl toward my fortigate :

admin@net-v-metrics01:/opt/fortigate_exporter$ curl localhost:9710/probe?target=https://192.168.100.254
# HELP probe_duration_seconds How many seconds the probe took to complete
# TYPE probe_duration_seconds gauge
probe_duration_seconds 0.002355984
# HELP probe_success Whether or not the probe succeeded
# TYPE probe_success gauge
probe_success 0

My fortigate is on v7.0.12 and its IP is 192.168.100.254. Prometheus server on which fortiage_exporter runs has IP 192.168.100.100. Server can ping fortigate without any issue, and when I am doing diagnose sniffer packet capture on Fortigate I can see communication between prometheus server and Fortigate back and forth on port 443.

Here's below my auth file :

"https://192.168.100.254":
  token: xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx

And here's section in prometheus.yml related to fortigate_exporter :

  - job_name: 'fortigate_exporter'
    metrics_path: /probe
    static_configs:
      - targets:
        - https://192.168.100.254
    relabel_configs:
      - source_labels: [__address__]
        target_label: __param_target
      - source_labels: [__param_target]
        target_label: instance
        # Drop the https:// and port (if specified) for the 'instance=' label
        regex: '(?:.+)(?::\/\/)([^:]*).*'
      - target_label: __address__
        replacement: 192.168.100.100:9710

Thanks in advance for your help ;)

Best Regards,

Adrien

[DevDorrejo]

same case #276 i have but with container.

[Hummdis]

I'm having the same issue. Fortigate Exporter v1.24.1 with Fortinet OS v7.4.2. I get all the metrics and even cURL works:

# curl -I -X GET https://10.0.0.1:4443
HTTP/2 200 
content-encoding: gzip
content-type: text/html
etag: b6yn17gzxccjztNnf4fG0HsQsg7r4pfb
x-frame-options: SAMEORIGIN
content-security-policy: frame-ancestors 'self'
x-xss-protection: 1; mode=block
strict-transport-security: max-age=15552000
date: Mon, 29 Jan 2024 21:42:56 GMT

However, the probe_success still shows 0 when it would appear everything is actually working fine.

[greenspartan]

Hi guys,

Any feedback ?

I have just tested the rest API from the same server using the token and it works like a charm :

net-v-metrics01:~$ curl -k -i -X GET "https://192.168.100.254/api/v2/monitor/system/time" -H "Authorization: Bearer xxxxxxxxxxxxxxxxxxxxxxxxxx" 
HTTP/2 200 
date: Tue, 20 Feb 2024 16:52:43 GMT
x-frame-options: SAMEORIGIN
content-security-policy: frame-ancestors 'self'
x-xss-protection: 1; mode=block
cache-control: no-cache, must-revalidate
content-length: 225
content-type: application/json

{
  "http_method":"GET",
  "results":{
    "time":1708447963
  },
  "vdom":"root",
  "path":"system",
  "name":"time",
  "action":"",
  "status":"success",
  "serial":"FG180FTKxxxxxxx",
  "version":"v7.0.12",
  "build":523

So it really seems issue is not coming from the fortigate.

More interesting, when I perform diagnose debug application httpsd -1 on the fortigate at the same time i am doing :

curl 'localhost:9710/probe?target=https://192.168.100.254'

I can see nothing poping up in fortigate httpsd debug, while i can perfectly see my attempts when I am using curl command directly from the server (not via fortigate_exporter).

Is it possible that fortigate_exporter is not querying correctly the fortigate ? Is it more logs i can get from fortigate_exporter to understand the issue ?

Thanks in advance for your help !

Best Regards,

Adrien

[Hummdis]

So, here's what I've just found, now that I've had a chance to really dig into this:

Apr 02 16:49:56 localhost fortigate_exporter[23193]: 2024/04/02 16:49:56 Error: Response code was 424, expected 200 (path: "api/v2/monitor/log/fortianalyzer")
Apr 02 16:49:56 localhost fortigate_exporter[23193]: 2024/04/02 16:49:56 Error: Response code was 404, expected 200 (path: "api/v2/monitor/switch-controller/managed-switch")

Upon seeing the above, I had to add the following probe exclusions to the Fortigate Exporter's list:

  probes:
    exclude:
      # Bug: https://github.com/bluecmd/fortigate_exporter/issues/277
      - Switch/ManagedSwitch
      - Log/Fortianalyzer/Status
      - Log/Fortianalyzer/Queue
      # End of bug fix.

This now shows the probe_success as 1 like it should. Therefore, something has become broken in the Fortianalyzer that breaks this, but I'm not sure what. I believe the exclusions to be a work around, not a solution.

[greenspartan]

Hi @Hummdis ,

Thanks for the feedback ! Can you let me know in which file you are confuring those probe exclusions ? Is it in prometheus.yml ?

Thanks !

Best Regards,

Adrien

[Hummdis]

It's in the fortigate-key.yaml file where you put your API token for each Fortigate that's being monitored.