opnsense / core

OPNsense GUI, API and systems backend
https://opnsense.org/
BSD 2-Clause "Simplified" License
3.26k stars 726 forks source link

New Dashboard Widget "URL Table IP Summary" and show info about URL Table IPs #5223

Closed nzkiwi68 closed 2 years ago

nzkiwi68 commented 3 years ago

Important notices Before you add a new report, we ask you kindly to acknowledge the following:

Is your feature request related to a problem? Please describe. Sort of. I use a number of URL Table IP aliases and fetch all sorts of thing, such as;

I then use 2 floating rules, one to drop from source and one to drop to destination for protection.

The issue is, if anything breaks, if a list changes and no longer updates, or, worse it's empty, you just don't easily know.

It's a very manual manual process to go into Firewall > Diagnostic > Aliases and then select each URL tables alias in turn to check if it has any IP addresses and then you still don't know when it was last updated.

Describe the solution you like

A Dashboard Widget that contained all of the URL Table IP aliases and showed their status Name, Total IP addresses, Refresh Frequency, last updated (date & time)

That way, you can easily at a glance see that the URL Table Aliases are functioning correctly (or not).

Describe alternatives you considered

There isn't one, only a very manual process.

nzkiwi68 commented 3 years ago

I hate to mention pfBlocker, but, such a widget will go a long way to making the "URL Table IPs" more useful and more functional.

pfBlocker has a Dashboard widget and that's how you can see easily if it's working correctly.

AdSchellevis commented 3 years ago

Extending the Firewall > Diagnostic > Aliases view with some information to ease troubleshooting might be something to consider, most people use a lot of aliases which doesn't really present well in a widget.

Some sort of log triggered notification might be practical at some point in time as well, but is more or less a generic feature, usually all relevant information is already in there.

nzkiwi68 commented 3 years ago

I'm not suggesting the widget showed all aliases, but, specifically only "URL Table (IPs)". I imagine you're right that others use lots of aliases, but lots of "URL Table (IPs)" aliases? I do use "URL Table IPs" but only a few and certainly not lots.

Have a look at a working pfSense firewall with the pfBlockerNG dashboard widget. It is readily apparent that all the IP lists are downloaded because the "Count" is not zero and "Updated" column showing a date and time last updated.

I found recently on a production OPNsense installation by chance that one "URL Table (IPs)" was incorrectly empty. In any event, even if it did contain IPs there's no easy way to tell when it was last downloaded successfully.

pfBlocker Widget: pfBlocker .

AdSchellevis commented 3 years ago

Just saying we don't consider adding a dashboard widget, if someone does want to work on such a feature, I don't mind that much, it could be a plugin as well.

Failures during updates should be in the log, when a list contains items, it doesn't guarantee a successful load, depends on how items are being flushed and what happens after download failures (often the previous state stays in the table).

nzkiwi68 commented 3 years ago

....when a list contains items, it doesn't guarantee a successful load, depends on how items are being flushed and what happens after download failures....

And that is essentially what such a widget does. It shows in a simple format that the IP address lists are not empty and have been updated. This gives you the confidence that your URL Table (IPs) aliases are working correctly.

It may well be logged, but it's just totally impractical to troll through the log to ensure your URL table IPs are up to date.

It's that widget that makes pfBlocker so darn useful, in a single glance I can tell if all is well or not.

Perhaps someone who writes code will pick this idea up. Maybe...

AdSchellevis commented 3 years ago

Perhaps someone who writes code will pick this idea up. Maybe...

you never know, but improving the current diagnostics view might be a good idea as well.

nzkiwi68 commented 3 years ago

I'm no programmer, but if you're going to go to effort of writing code that pulls URL Tables (IPs) into a list and displays them as; Name, Total IP addresses, Refresh Frequency, last updated (date & time)

How much extra effort is that to make that view available as widget?

AdSchellevis commented 3 years ago

I was think maybe we could add the number of entries and last updated in the current dropdown to keep the current overview consistent, otherwise the whole page should be refactored (or extended with some sort of "overview" tab).

nzkiwi68 commented 3 years ago

Yes, sounds good, certainly an improvement.

Still, terribly manual to need to specifically drill down to diagnostics to see if the URL Table IPs are working. Maybe later on someone can pickup that code and make it into a widget...

kulikov-a commented 3 years ago

as an immediate help, can suggest adding the pfctl -vvsTables output to the pfInfo page (just a little code to the page and pfinfo.py script). this allows to take a look at the tables on the subject of health and stats. small problem: the output only displays the time the table was last cleared, and the tables are not flushed before reloading. but I have a feeling that @AdSchellevis has already started adding this info to another script )

AdSchellevis commented 3 years ago

@kulikov-a I was indeed thinking about this a bit, the most logical place seems to be the current alias grid, this is heading into the master branch image

kulikov-a commented 3 years ago

@AdSchellevis an ideal place imho

nzkiwi68 commented 3 years ago

Thanks btw for picking up this idea....

OPNsense-bot commented 2 years ago

This issue has been automatically timed-out (after 180 days of inactivity).

For more information about the policies for this repository, please read https://github.com/opnsense/core/blob/master/CONTRIBUTING.md for further details.

If someone wants to step up and work on this issue, just let us know, so we can reopen the issue and assign an owner to it.