Added gateway statuses

[FreeWillyB]

I wanted to be able to see if pfSense had marked a gateway as down or not, so I created this script to add that functionality. It provides the same data points as you originally add with the same field names, but I've added the gateway IP, monitor IP, status, and substatus returned from the "return_gateways_status_text" function from "/etc/inc/gwlb.inc" PHP script.

I also modified the dashboard and added a "gateway summary" table. Even if the user isn't running this plugin, the table will still show the Gateway Name, loss, and RTT values.

[VictorRobellini]

Thanks for the PR. I'll merge later today. Can you attach a screenshot of the updated widget here?

[FreeWillyB]

No problem! But I won't be able to get it to you until Monday.

[FreeWillyB]

Here's that screenshot. I also made a couple more tweaks, including the Title of the dashboard that I had changed in order to add it to my grafana. I also added a blurb about the "Filer" add-on package for pfSense that makes it a breeze to add/edit the plugins.

[VictorRobellini]

Substatus

status changed to be one of online or down. substatus can be one of none, down, highloss, highlatency, loss, latency, or force_down

Who knew?! I looked through things and I only have 1 change request and I'm pretty stupid when it comes to git. Your script to pull the gateway status is python calling PHP, parsing the output, and then reformatting it for telegraf. I'm no PHP developer but I took a stab at a native PHP example. I haven't run this in my telegraf set up yet but I would prefer a native approach (less tendency to break as things get upgraded)

#!/usr/local/bin/php-cgi -f
<?php
require_once("config.inc");
require_once("gwlb.inc");

$host = gethostname();
$source = "pfconfig";

$gwstat = return_gateways_status($byname);

foreach ($gwstat as $gw_ip => $gwaddress) {
    $gateway = $gw_ip;
    $monitor = $gwstat[$gw_ip]["monitorip"];
    $source = $gwstat[$gw_ip]["srcip"];
    $delay = $gwstat[$gw_ip]["delay"];
    $stddev = $gwstat[$gw_ip]["stddev"];
    $loss = $gwstat[$gw_ip]["loss"];
    $status = $gwstat[$gw_ip]["status"];
    $substatus = $gwstat[$gw_ip]["substatus"];

    # Some error checking here
    #if (!isset($xxx)){
    #    $xxx = "Unkown";
    #}

    printf("gateway,host=%s,gateway=%s,monitor=%s,source=%s delay=%s stddev=%s loss=%s status=%s substatus=%s\n",
        $host,
        $gateway,
        $monitor,
        $source,
        floatval($delay),
        floatval($stddev),
        floatval($loss),
        $status,
        $substatus,
    );
}
?>

[FreeWillyB]

Sorry didn’t meant to send that last email. I tried something line that but I seem to recall having some issues with PHP, I think it was something to do with formatting for telegraf? But I’ll look at this script and test it out on my set up when I get a chance this week to see how it works.

Also, I didn’t get to play with formatting the status codes beyond a simple change of “none => OK” so you might want to look at that. Also, I just remember I think I changed or removed some other thresholds for my personal use. I’m not sure they got put back. 😳 you might want to just look at the cards I changed themselves rather than the whole dashboard JSON.

[VictorRobellini]

The php I pasted above works, I just don't know enough about what gets returned to know if we need to update a null return. The only thing it's missing is the interface name. I'll sort that out and update.

[VictorRobellini]

I guess I can also replace the other gateway script with this as well. 👍

[FreeWillyB]

That's a good point, since it wouldn't matter what python version anymore.

The only thing I see up front just from looking on my phone is that you are using "delay" for the tag instead of "rtt", so that would break any previous tag values in the database.

As far as null values go, I'm not sure what would return that. I will have to look at a couple scenarios with disabled gateways or something to see if they return weird values.

[VictorRobellini]

I found more data to pull in from the pfsense PHP page.

#!/usr/local/bin/php-cgi -f
<?php
require_once("config.inc");
require_once("gwlb.inc");

$host = gethostname();
$source = "pfconfig";

$a_gateways = return_gateways_array();
$gateways_status = return_gateways_status(true);

foreach ($a_gateways as $i => $gateway) {

    $name = $gateways_status[$i]["name"];
    $monitor = $gateways_status[$i]["monitorip"];
    $source = $gateways_status[$i]["srcip"];
    $delay = $gateways_status[$i]["delay"];
    $stddev = $gateways_status[$i]["stddev"];
    $loss = $gateways_status[$i]["loss"];
    $status = $gateways_status[$i]["status"];
    $substatus = $gateways_status[$i]["substatus"];

    $interface = $gateway["interface"];
    $friendlyname = $gateway["friendlyiface"]; # This is not the friendly interface name so I'm not using it
    $friendlyifdescr = $gateway["friendlyifdescr"];
    $gwdescr = $gateway["descr"];
    $defaultgw = $gateway['isdefaultgw'];

    if (!isset($gateway['isdefaultgw'])) {
        $defaultgw = "1";
    } else {
        $defaultgw = "0";
    }

    if ($gateways_status[$i]) {
        if (isset($gateway['monitor_disable'])) {
            $monitor = "Unmonitored";
            $delay = "Pending";
            $stdev = "Pending";
            $loss = "Pending";
        }
    }

    printf("gateways,host=%s,name=%s,gateway=%s,monitor=%s,source=%s,friendlyname=%s,interface=%s gwdescr=\"%s\",delay=%s,stddev=%s,loss=%s,status=\"%s\",substatus=\"%s\"\n",

        $host,
        $name,
        $defaultgw,
        $monitor,
        $source,

        $friendlyifdescr,

        $interface,

        # $friendlyifdescr,
        $gwdescr,
        floatval($delay),
        floatval($stddev),
        floatval($loss),
        $status,
        $substatus
    );

};
?>

[FreeWillyB]

Nice! I was able to get the friendly name by calling "return_gateway_status(true)", but the interface and "is default GW" could be useful. But is there a real use case for returning the GW/IF descriptions? Seems like it would just fill the database with unnecessary data every 10 seconds, especially when the user will likely already know what it should be. At least that's kinda my take on it anyways.

[FreeWillyB]

I pushed my changes with the PHP script, so I think they should be in that PR. I can add the interface and default GW status as well.

Also, I meant to mention that I tested my previous commit with a test gateway and it returned valid data for whether it was forced down, disabled, or just monitored was disabled.

[FreeWillyB]

I also just added the extra features you were mentioning. The only thing I didn't input into the database was the gateway description. Everything else like the interface (igb0), friendly name (opt1), and interface description (WAN) is now included in the plugin. There's a lot of information in there now, and it would honestly warrant a full-width table. I'll work on tweaking the dashboard details later, including setting the thresholds for the different columns.

[FreeWillyB]

Also, I just noticed based on the "substatus" link you sent earlier, but some versions of pfSense will not return the substatus. My latest commit handles that checking if that value is set and then manually sets it to "N/A".

[VictorRobellini]

I think that any additional update is going to require an additional PR. I had to modify the dashboard to organize the graphs. When I imported the dashboard from your PR, the "Network Stats" section was empty and all of it's graphs had moved to a different section. I also updated the queries to take into account the gateway filter/dropdown at the top of the dashboard and updated the dashboard variable.

I also updated the PHP script to move most of the data from tags to fields. I'm making this up as I go, but if it's not going to be used by a group by function, I make it a field.

I would like to change the color of the status/substatus fields based on their value, but I don't think it's possible in Grafana with a string.