revisit default alerting and guessed types/icons

FlyingToto commented 3 months ago

Is there an existing issue for this?

[X] I have searched the existing open and closed issues

Am I willing to test this? 🧪

[X] I will do my best to test this feature on the netlertx-dev image when requested within 48h and report bugs to help deliver a great user experience for everyone and not to break existing installations.

Can I help implement this? 👩‍💻👨‍💻

[X] Yes
[X] No

Is your feature request related to a problem? Please describe

Not an issue aside from human laziness... it can be a bit tedious to customize a lot of params manually.. basically, I think we could enhance NaX guesses in terms of types/icons and from there refine alerting...

Describe the solution you'd like

taking a multi-steps approach:

Best guess unknown device types (and logo) by:

routing details (we already identify the ISP gateway throrugh dig, we can identify routers and/or firewalls from traceroute/interfaces settings, )
services (leveraging nmap/nbdt/... ) we might be able to identify a server vs win pc vs linux vs a phone?
from the MAC address, we can most likely make an educated guess from the Vendor MAC (unifi,cisco,...=switch/ap/router), irobot 80% chance= vaccuum,... )
vendor logo and family-generic type ie: if MAC=amazon then logo=amazon, type=IOT (people can then refine it between speaker, TVstreamer, doorbell... )
use a vendor logo and type=undefined
use ? logo and type=unknown as a last resort

based on best practices and guessed types pre-configure default alerts...

for insance:

rogue DHCP Server -> call the cops & army
if new MAC detected for the first time-> notify by texts, phone calls,
if router/gateway goes down -> notify by texts,....
if switch/servers/IOT goes down -> notify by email
everything else, generate weekly report by email... etc....

Describe alternatives you've considered

for the time being, I have been using the device screens and sorting to select devices by types or by MAC and manually mult-edit them accordingly... that works but takes a bit of time....

Anything else?

maybe we could setup a google spreadsheet over the internet to ask users to fill in their preferences in terms of guessed types/logos/alerts.... basically crowdsource it...

jokob-sk commented 2 months ago

Hey @FlyingToto ,

Thanks for the suggestion. I was thinking about this for the last 2 days.

Currently, I don't plan to implement auto-notification setups as there are a lot of notification-related settings, and guessing defaults for different users and use cases is difficult. It also helps users discover customizations of the app, so it might be counter-productive from a learning-the-app perspective.

Also, I think a lot of people would have different opinions about what should be a default and what not, and what should be configurable. Currently, it's also difficult to have an overridable default setting.

Regarding icons - I implemented a basic guess_icon method Regarding types - I implemented a basic guess_type method

Let's see if that creates too much overhead on the DB as it may introduce DB_Locked errors if too much is going on during scans.

FlyingToto commented 2 months ago

the guess_icon and guess_type methods only need to be called the first time the device is discovered isn't it? or maybe the first time a given plugin reports on a given device. it could also be run out of cycle, int it?

jokob-sk commented 2 months ago

Yes, but even with that one call I noticed some locked DB errors. Let's see if it manifests further. In general, the whole device.py file could do with some optimizations on DB access. https://github.com/jokob-sk/NetAlertX/blob/main/server/device.py

I'm a bit pressed with time the next few weeks, so let's see if I get the energy/time to have a look.

FlyingToto commented 2 months ago

neat! I will play with it... would it make a difference if you were to store guessed types and guessed icons in lists indexed by MAC vendors? that way you don't have to go through the whole list if elif elif... (I would assume that retrieving a value from an index array requires python to calculate a hash which is likely to take more time)

FlyingToto commented 2 months ago

wow, actually I stand corrected... according to claude.ai (and I checked in testre because I don't trust the thing) using a dictionary is actually much faster even with small list of 5 choices.... about 3x faster!

jokob-sk commented 2 months ago

Oh, interesting - sorry missed the replies here.

The only way I could keep the application running without too many DB-locked errors is to implement an API layer for the front end. This could still be refined (e.g. only update the API endpoints when requested/needed instead of doing this periodically or with every change. These kinds of improvements decrease the amount of IO operations more than low-level code optimization, which of course helps as well, just the API layer is used more often.

FlyingToto commented 2 months ago

I would think that the only time you would call this guess_icon/guess_types is when a brand new device gets identified which doesn't already have a type or icon...
For instance, when the omada plugin runs, it will already set the type for switches, access-points and firewall/gateway. so in this case the only thing missing is the icon, (and actually I could populate it as well except that I am running out of variables)

furthermore,

the type and icon would most likely remain constant for a given MAC address... so for instance if a device gets deleted, then recreated later on, as long as it has the same MAC, you can reuse the same icon/type.
the other advantage is that If someone manually sets the type/icon to different value you don't want to have them reset... so it should definitely stick.

one question:

how to do you identify a device with random MAC? (and are you able to realize that these 2 MAC overtime, are actually the same device that keeps randomly changing?)

jokob-sk commented 2 months ago

Thanks for the reply!

I actually already implemented the guessing of icons, you can check the above commit :)

https://github.com/jokob-sk/NetAlertX/commit/54b6b1d408b04733b52fac9449719d7b2041463e

How random MACs are determined is described here: https://github.com/jokob-sk/NetAlertX/blob/main/docs/RANDOM_MAC.md

FlyingToto commented 2 months ago

yes I saw you already have both methods, I think we can tweak it a bit... I have been playing on my side.

jokob-sk commented 2 months ago

Great, looking forward what you come up with 🙂 I just pushed the auto-network tree creation in the UNIFI plugin, same as what you did in the OMADA plugin, based on the groundwork done when we did it on your plugin.

jokob-sk commented 1 month ago

Releasing -> closing

jokob-sk / NetAlertX