search

bluecmd / fortigate_exporter

Prometheus exporter for Fortigate firewalls
GNU General Public License v3.0
229 stars 71 forks source link

probin api error #91

Closed andrewm659 closed 3 years ago

andrewm659 commented 3 years ago

I am trying to probe my fortigate 60D and getting the following errors:


2021/05/10 12:58:30 Error: Response code was 429, expected 200 (path: "api/v2/monitor/system/status")
2021/05/10 12:58:30 Error: Response code was 429, expected 200 (path: "api/v2/monitor/system/resource/usage")
2021/05/10 12:58:30 Error: Response code was 429, expected 200 (path: "api/v2/monitor/system/resource/usage")
2021/05/10 12:58:30 Error: Response code was 429, expected 200 (path: "api/v2/monitor/firewall/policy/select")
2021/05/10 12:58:30 Error: Response code was 429, expected 200 (path: "api/v2/monitor/system/interface/select")
2021/05/10 12:58:30 Error: Response code was 429, expected 200 (path: "api/v2/monitor/vpn/ssl")
2021/05/10 12:58:30 Error: Response code was 429, expected 200 (path: "api/v2/monitor/vpn/ipsec")
2021/05/10 12:58:30 Error: Response code was 429, expected 200 (path: "api/v2/monitor/system/ha-statistics")
2021/05/10 12:58:30 Error: Response code was 429, expected 200 (path: "api/v2/monitor/license/status/select")
2021/05/10 12:58:30 Error: Response code was 429, expected 200 (path: "api/v2/monitor/system/link-monitor")
2021/05/10 12:58:30 Error: Response code was 429, expected 200 (path: "api/v2/monitor/virtual-wan/health-check")
2021/05/10 12:58:31 Error: Response code was 429, expected 200 (path: "api/v2/monitor/system/available-certificates")
2021/05/10 12:58:31 Error: Response code was 429, expected 200 (path: "api/v2/monitor/firewall/load-balance")
2021/05/10 12:58:31 Probe of "https://1.2.3.4.:7443" failed, took 1.512 seconds

Is this related to the httpd issue mentioned in the info page?

bluecmd commented 3 years ago

Hi! No, 429 is an login issue - it means you have the wrong credentials.

andrewm659 commented 3 years ago

hmmm.....that's weird. I got the API key from the user I created/generated it with. Is there something else i'm doing wrong maybe? Something in the connection string? Also thank for this exporter!

andrewm659 commented 3 years ago

I just regenerated the api key and changed the profile associated with it. Still getting same error message

andrewm659 commented 3 years ago

How do I specify a username? It keeps defaulting to Administrator and I can't generate a api key for the admin user.

bluecmd commented 3 years ago

The API key does not have a username, it is global - so there is none. I am not sure what is going wrong in your case.

What I could suggest you try is to ensure the key works when you use e.g. curl. Like this: curl -k https://<deviceipaddress>/api/v2/cmdb/firewall/address?vdom=root&access_token=<apikeytokenhere>

andrewm659 commented 3 years ago 
[ameyer@mon001 ~]$ curl -ilk https://192.168.1.1:7443/api/v2/cmdb/firewall/address?vdom=root&access_token=
[1] 2487587
[ameyer@mon001 ~]$ HTTP/1.1 401 Unauthorized
Date: Mon, 10 May 2021 19:22:34 GMT
Server: Apache
Content-Length: 503
Content-Type: text/html; charset=iso-8859-1

<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html><head>
<title>401 Unauthorized</title>
</head><body>
<h1>Unauthorized</h1>
<p>This server could not verify that you
are authorized to access the document
requested.  Either you supplied the wrong
credentials (e.g., bad password), or your
browser doesn't understand how to supply
the credentials required.</p>
<p>Additionally, a 401 Unauthorized
error was encountered while trying to use an ErrorDocument to handle the request.</p>
</body></html>

[1]+  Done                    curl -ilk https://10.150.1.1:7443/api/v2/cmdb/firewall/address?vdom=root
[ameyer@mon001 ~]$
bluecmd commented 3 years ago

Well, that should work - I would say that either Fortigate is incorrectly configured, or your middleware (Apache in this case it seems) might be doing something.

andrewm659 commented 3 years ago

I tried without the vdom_root, still no dice. I posted something to the fortinet fuse community. BTW, i'm using FortiOS 6.2.7 for my Fortinet 60D.

andrewm659 commented 3 years ago

So I was able to run through some of the examples on this blog post and they worked just fine. http://socpuppet.blogspot.com/2019/09/howto-use-fortios-apiuser.html

So i'm not sure what the issue is.

bluecmd commented 3 years ago

Ah right, when you tried with curl you need to quote your URL. Anyway, can you try with the access key as a header? Like --header 'Authorization: Bearer xyz' - that's the way the exporter probes. Just to make sure any proxy or middleman you have doesn't drop that header.

andrewm659 commented 3 years ago

Yes. I was successful in doing this. Also found out I was using version 1.3.0 and upgraded that. 


[ameyer@mon001 ~]$ curl -k  -H -- "Authorization: Bearer $API-KEY"  "https://10.150.1.1:7443/api/v2/cmdb/system/global?access_token=$API_KEY"
curl: (3) Port number ended with ' '
{
  "http_method":"GET",
  "revision":"",
  "results":{
    "language":"english",
    "gui-ipv6":"disable",
    "gui-certificates":"enable",
    "gui-custom-language":"disable",
    "gui-wireless-opensecurity":"disable",
    "gui-display-hostname":"disable",
    "gui-fortisandbox-cloud":"disable",
    "gui-lines-per-page":50,
    "admin-https-ssl-versions":"tlsv1-1 tlsv1-2 tlsv1-3",
    "admintimeout":5,
    "admin-console-timeout":0,
    "ssd-trim-freq":"weekly",
    "ssd-trim-hour":1,
    "ssd-trim-min":60,
    "ssd-trim-weekday":"sunday",
    "ssd-trim-date":1,
    "admin-concurrent":"enable",
    "admin-lockout-threshold":3,
    "admin-lockout-duration":60,
    "refresh":0,
    "interval":5,
    "failtime":5,
    "daily-restart":"disable",
    "restart-time":"00:00",
    "radius-port":1812,
    "admin-login-max":100,
    "remoteauthtimeout":5,
    "ldapconntimeout":500,
    "batch-cmdb":"enable",
    "multi-factor-authentication":"optional",
    "ssl-min-proto-version":"TLSv1-2",
    "autorun-log-fsck":"disable",
    "dst":"enable",
    "timezone":"08",
    "traffic-priority":"tos",
    "traffic-priority-level":"medium",
    "anti-replay":"strict",
    "send-pmtu-icmp":"enable",
    "honor-df":"enable",
    "revision-image-auto-backup":"disable",
    "revision-backup-on-logout":"disable",
    "management-vdom":"root",
    "hostname":"asmfw01",
    "gui-allow-default-hostname":"disable",
    "alias":"FortiGate-100D",
    "strong-crypto":"enable",
    "ssh-cbc-cipher":"enable",
    "ssh-hmac-md5":"enable",
    "ssh-kex-sha1":"enable",
    "ssh-mac-weak":"enable",
    "ssl-static-key-ciphers":"enable",
    "snat-route-change":"disable",
    "cli-audit-log":"disable",
    "dh-params":"2048",
    "fds-statistics":"enable",
    "fds-statistics-period":60,
    "tcp-option":"enable",
    "lldp-transmission":"disable",
    "lldp-reception":"disable",
    "proxy-auth-timeout":10,
    "proxy-re-authentication-mode":"session",
    "proxy-auth-lifetime":"disable",
    "proxy-auth-lifetime-timeout":480,
    "sys-perf-log-interval":5,
    "check-protocol-header":"loose",
    "vip-arp-range":"restricted",
    "reset-sessionless-tcp":"disable",
    "allow-traffic-redirect":"enable",
    "strict-dirty-session-check":"enable",
    "tcp-halfclose-timer":120,
    "tcp-halfopen-timer":10,
    "tcp-timewait-timer":1,
    "udp-idle-timer":180,
    "block-session-timer":30,
    "ip-src-port-range":"1024-25000",
    "pre-login-banner":"disable",
    "post-login-banner":"disable",
    "tftp":"enable",
    "av-failopen":"pass",
    "av-failopen-session":"disable",
    "memory-use-threshold-extreme":95,
    "memory-use-threshold-red":88,
    "memory-use-threshold-green":82,
    "cpu-use-threshold":90,
    "check-reset-range":"disable",
    "vdom-mode":"no-vdom",
    "vdom-admin":"",
    "long-vdom-name":"disable",
    "admin-port":80,
    "admin-sport":7443,
    "admin-https-redirect":"enable",
    "admin-hsts-max-age":15552000,
    "admin-ssh-password":"enable",
    "admin-restrict-local":"disable",
    "admin-ssh-port":22,
    "admin-ssh-grace-time":120,
    "admin-ssh-v1":"disable",
    "admin-telnet":"enable",
    "admin-telnet-port":23,
    "default-service-source-port":"1-65535",
    "admin-maintainer":"enable",
    "admin-server-cert":"self-sign",
    "user-server-cert":"Fortinet_Factory",
    "admin-https-pki-required":"disable",
    "wifi-certificate":"Fortinet_Wifi",
    "wifi-ca-certificate":"Fortinet_Wifi_CA",
    "auth-http-port":1000,
    "auth-https-port":1003,
    "auth-keepalive":"disable",
    "policy-auth-concurrent":0,
    "auth-session-limit":"block-new",
    "auth-cert":"Fortinet_Factory",
    "clt-cert-req":"disable",
    "fortiservice-port":8013,
    "cfg-save":"automatic",
    "cfg-revert-timeout":600,
    "reboot-upon-config-restore":"enable",
    "admin-scp":"disable",
    "security-rating-result-submission":"enable",
    "security-rating-run-on-schedule":"enable",
    "internal-switch-mode":"interface",
    "internal-switch-speed":"",
    "wireless-controller":"enable",
    "wireless-controller-port":5246,
    "fortiextender-data-port":25246,
    "fortiextender":"enable",
    "fortiextender-vlan-mode":"disable",
    "switch-controller":"enable",
    "switch-controller-reserved-network":"169.254.0.0 255.255.0.0",
    "dnsproxy-worker-count":1,
    "url-filter-count":1,
    "proxy-worker-count":0,
    "scanunit-count":0,
    "proxy-kxp-hardware-acceleration":"enable",
    "proxy-cipher-hardware-acceleration":"enable",
    "fgd-alert-subscription":"",
    "ipsec-hmac-offload":"enable",
    "ipv6-accept-dad":1,
    "ipv6-allow-anycast-probe":"disable",
    "csr-ca-attribute":"enable",
    "wimax-4g-usb":"disable",
    "cert-chain-max":8,
    "sslvpn-max-worker-count":0,
    "sslvpn-kxp-hardware-acceleration":"enable",
    "sslvpn-cipher-hardware-acceleration":"enable",
    "sslvpn-plugin-version-check":"enable",
    "two-factor-ftk-expiry":60,
    "two-factor-email-expiry":60,
    "two-factor-sms-expiry":60,
    "two-factor-fac-expiry":60,
    "two-factor-ftm-expiry":72,
    "virtual-server-count":0,
    "virtual-server-hardware-acceleration":"enable",
    "wad-worker-count":0,
    "wad-csvc-cs-count":1,
    "wad-csvc-db-count":0,
    "wad-source-affinity":"enable",
    "wad-memory-change-granularity":10,
    "login-timestamp":"disable",
    "miglogd-children":0,
    "special-file-23-support":"disable",
    "log-uuid-policy":"enable",
    "log-uuid-address":"enable",
    "log-ssl-connection":"disable",
    "arp-max-entry":131072,
    "ndp-max-entry":0,
    "br-fdb-max-entry":8192,
    "max-route-cache-size":0,
    "ipsec-asic-offload":"enable",
    "ipsec-soft-dec-async":"disable",
    "device-idle-timeout":300,
    "gui-device-latitude":"",
    "gui-device-longitude":"",
    "private-data-encryption":"disable",
    "auto-auth-extension-device":"enable",
    "gui-theme":"green",
    "gui-date-format":"yyyy\/MM\/dd",
    "gui-date-time-source":"system",
    "igmp-state-limit":3200,
    "cloud-communication":"enable",
    "fec-port":50000,
    "fortitoken-cloud":"enable"
  },
  "vdom":"root",
  "path":"system",
  "name":"global",
  "status":"success",
  "http_status":200,
  "serial":"",
  "version":"",
  "build":
}
bluecmd commented 3 years ago

Can you try the same without the API key in the URL (so it is only in the header)?

andrewm659 commented 3 years ago

I removed the API key and got this:

[ameyer@mon001 ~]$ curl -k  -H -- "Authorization: Bearer API_KEY"  "https://10.150.1.1:7443/api/v2/cmdb/system/global?access_token="
curl: (3) Port number ended with ' '
<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html><head>
<title>401 Unauthorized</title>
</head><body>
<h1>Unauthorized</h1>
<p>This server could not verify that you
are authorized to access the document
requested.  Either you supplied the wrong
credentials (e.g., bad password), or your
browser doesn't understand how to supply
the credentials required.</p>
<p>Additionally, a 401 Unauthorized
error was encountered while trying to use an ErrorDocument to handle the request.</p>
</body></html>
[ameyer@mon001 ~]$
bluecmd commented 3 years ago

This is what I mean: curl -k -H "Authorization: Bearer $API_KEY" "https://10.150.1.1:7443/api/v2/cmdb/system/global

That cURL command should be very close to what the exporter does. I have confirmed that it works for me against my fortigates.

aaron791109 commented 3 years ago

curl -k -H -- "Authorization: Bearer $API-KEY" "https://fortigate/api/v2/cmdb/system/global?access_token=$API_KEY

I can get the data from this CURL. However, I got the result with "code 401 & 429" when putting the "$API_KEY" into "fortigate-key.yml". It shows "too_many_attempts 429 (USER=$API_KEY) " on the "fortigate log".

401 error = login failed from http(10.100.10.225) because of an internal error

I have no idea how to fix this problem.

bluecmd commented 3 years ago

You keep adding the access_token as a URL parameter, that is not what fortigate_exporter does - and the API should work without it (but you have to set it in the Authorization header). Please do not set both when debugging this, as it is impossible to tell which access_token field is being used.

Anyway, my bet is that the Apache server you are using (as I mentioned https://github.com/bluecmd/fortigate_exporter/issues/91#issuecomment-837198740) to proxy the Fortigate API is not forwarding the Authorization header.

bluecmd commented 3 years ago

I will close this right now as I cannot see this being an issue with the exporter - it seems to me that there is an extra proxy in the middle intercepting and modifying the traffic which is out of our control. That's what I can make of the available data right now anyway.