search

leiweibau / Pi.Alert

Scan the devices connected to your WIFI / LAN and alert you the connection of unknown devices. It also warns if a "always connected" device disconnects. In addition, it is possible to check web services for availability. For this purpose HTTP status codes and the response time of the service are evaluated.
https://leiweibau.net
GNU General Public License v3.0
453 stars 32 forks source link

[Bug] Problems with update process #379

Closed modem7 closed 1 month ago

modem7 commented 2 months ago

Is there an existing issue for this?

Current Behavior

Issues with current update. Throws up errors.

Pi.Alert v2024-09-01

Pi.Alert v2024-09-01
---------------------------------------------------------
Executing user: modem7

Update HW Vendors
    Timestamp: 2024-09-02 03:03:00

Updating vendors DB...
Traceback (most recent call last):
  File "/home/modem7//pialert/back/pialert.py", line 3609, in <module>
    sys.exit(main())
             ^^^^^^
  File "/home/modem7//pialert/back/pialert.py", line 94, in main
    res = update_devices_MAC_vendors('-s')
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/modem7//pialert/back/pialert.py", line 627, in update_devices_MAC_vendors
    update_output = subprocess.check_output (update_args)
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/subprocess.py", line 466, in check_output
    return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/subprocess.py", line 571, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['sh', '/home/modem7/pialert/back/update_vendors.sh', '-s']' returned non-zero exit status 1.

- Testing Pi.Alert Internet IP Lookup...

Pi.Alert v2024-09-01
---------------------------------------------------------
Executing user: modem7

Check Internet IP
    Timestamp: 2024-09-02 03:03:00

Retrieving Internet IP...
    myIP

Retrieving previous IP...
    myIP
    No changes to perform

Skipping Dynamic DNS update...

Continuous New Device Notification...
    Skipping... Not activated!

Auto Speedtest...
    Skipping Speedtest... Not activated!

Auto Update-Check...
    Crontab: 0 3,9,15,21 * * *
    Version Check function was NOT executed.

Auto Backup...
    Skipping Auto Backup... Not activated!

DONE!!!

- Testing Pi.Alert Network scan...
*** PLEASE WAIT A COUPLE OF MINUTES...
sudo: arp-scan: command not found
sudo: arp-scan: command not found
sudo: arp-scan: command not found
sudo: arp-scan: command not found
sudo: arp-scan: command not found
/usr/lib/python3/dist-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host '192.168.0.200'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings
  warnings.warn(
grep: /usr/share/arp-scan/ieee-oui.txt: No such file or directory

Pi.Alert v2024-09-01
---------------------------------------------------------
Executing user: modem7

Scan Devices
    Timestamp: 2024-09-02 03:03:00

Scanning...
    arp-scan Method...
    arp-scan: Multiple interfaces
    Pi-hole 5 Method...
        ...Skipped
    DHCP Leases Method...
        ...Skipped
    Fritzbox Method...
        ...Skipped
    Mikrotik Method...
        ...Skipped
    UniFi Method...
    Satellite Import...
        ...Skipped

Processing scan results...
    Processing ignore list...
        No MAC-Ignore list defined
        IP-Ignore list is empty
    Devices Detected.......: 39
        arp-scan Method....: 0
        Pi-hole Method.....: +0
        Fritzbox Method....: +0
        Mikrotik Method....: +0
        UniFi Method.......: +38
        New Devices........: 0

    Devices in this scan...: 39
        Down Alerts........: 7
        New Down Alerts....: 6
        New Connections....: 1
        Disconnections.....: 8
        IP Changes.........: 1

Updating DB Info...
    Sessions Events (connect / discconnect) ...
    Creating new devices...
    Updating Devices Info...
        Trying to resolve devices without name.......
        Names updated:   0
    Voiding false (ghost) disconnections...
    Pairing session events (connection / disconnection) ...
    Creating sessions snapshot...
    Skipping repeated notifications...
    Calculate Activity History...

Reporting...
    Formating report...
    Skip mail...
    Skip PUSHSAFER...
    Skip PUSHOVER...
    Skip Telegram...
    Sending report by NTFY...
    Save report to file...
    Notifications: 11

DONE!!!

------------------------------------------------------------
 Update process finished
------------------------------------------------------------

Environment

Static hostname: PiAlert Virtualization: lxc Operating System: Ubuntu 24.04.1 LTS Kernel: Linux 6.8.12-1-pve Architecture: x86-64

Anything else?

No response

leiweibau commented 2 months ago

Thanks for the info. I'll have a look at it.

leiweibau commented 2 months ago

I have run the update process on a Debian 9, a Linux Mint 22 and an Ubuntu Server LTS 22.04 and have not been able to reproduce the error so far. I will test other distributions.

modem7 commented 2 months ago

Are you able to try on Ubuntu LTS 24.04?

On 22.04, I don't seem to have any issues, but as soon as I upgrade to 24.04.1, I seem to get the issues.

On 22.04: python version = Python 3.11.6

On 24.04: python version = Python 3.12.3

Before the pialert upgrade (but post 24.04 upgrade), if I do ❯ python3 $HOME/pialert/back/pialert.py 1 I get:

Traceback (most recent call last):
  File "/home/modem7//pialert/back/pialert.py", line 21, in <module>
    from mac_vendor_lookup import MacLookup
ModuleNotFoundError: No module named 'mac_vendor_lookup'

And if I do the upgrade again, as above I get:

Configured Pi.Alert scans are enabled
- Testing Pi.Alert HW vendors database update process...
*** PLEASE WAIT A COUPLE OF MINUTES...
/home/modem7/pialert/back/update_vendors.sh: 20: cd: can't cd to /usr/share/ieee-data/
sudo: get-oui: command not found

Pi.Alert v2024-09-01
---------------------------------------------------------
Executing user: modem7

Update HW Vendors
    Timestamp: 2024-09-05 03:23:00

Updating vendors DB...
Traceback (most recent call last):
  File "/home/modem7//pialert/back/pialert.py", line 3609, in <module>
    sys.exit(main())
             ^^^^^^
  File "/home/modem7//pialert/back/pialert.py", line 94, in main
    res = update_devices_MAC_vendors('-s')
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/modem7//pialert/back/pialert.py", line 627, in update_devices_MAC_vendors
    update_output = subprocess.check_output (update_args)
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/subprocess.py", line 466, in check_output
    return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/subprocess.py", line 571, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['sh', '/home/modem7/pialert/back/update_vendors.sh', '-s']' returned non-zero exit status 1.

Post update, I get:

❯ python3 $HOME/pialert/back/pialert.py 1

Pi.Alert v2024-09-01
---------------------------------------------------------
Executing user: modem7

Scan Devices
    Timestamp: 2024-09-05 03:24:00

Scanning...
    arp-scan Method...
    arp-scan: Multiple interfaces
sudo: arp-scan: command not found

sudo: arp-scan: command not found

sudo: arp-scan: command not found

sudo: arp-scan: command not found

sudo: arp-scan: command not found

    Pi-hole 5 Method...
        ...Skipped
    DHCP Leases Method...
        ...Skipped
    Fritzbox Method...
        ...Skipped
    Mikrotik Method...
        ...Skipped
    UniFi Method...
/usr/lib/python3/dist-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host '192.168.0.200'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings
  warnings.warn(
    Satellite Import...
        ...Skipped

Processing scan results...
    Processing ignore list...
        No MAC-Ignore list defined
        IP-Ignore list is empty
    Devices Detected.......: 40
        arp-scan Method....: 0
        Pi-hole Method.....: +0
        Fritzbox Method....: +0
        Mikrotik Method....: +0
        UniFi Method.......: +39
        New Devices........: 0

    Devices in this scan...: 40
        Down Alerts........: 6
        New Down Alerts....: 0
        New Connections....: 0
        Disconnections.....: 0
        IP Changes.........: 0

Updating DB Info...
    Sessions Events (connect / discconnect) ...
    Creating new devices...
    Updating Devices Info...
grep: /usr/share/arp-scan/ieee-oui.txt: No such file or directory
        Trying to resolve devices without name.......
        Names updated:   0
    Voiding false (ghost) disconnections...
    Pairing session events (connection / disconnection) ...
    Creating sessions snapshot...
    Skipping repeated notifications...
    Calculate Activity History...

Reporting...
    Formating report...
    No changes to report...
    Notifications: 11

DONE!!!

So, looking further, it seems that the arp-scan package gets removed/superseded/changed/whatever, and needs reinstalling. 

❯ sudo apt-get install -y arp-scan
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
The following additional packages will be installed:
  ibverbs-providers ieee-data libibverbs1 libnl-route-3-200 libpcap0.8t64
The following NEW packages will be installed:
  arp-scan ibverbs-providers ieee-data libibverbs1 libnl-route-3-200 libpcap0.8t64
0 upgraded, 6 newly installed, 0 to remove and 0 not upgraded.
Need to get 1274 kB/3387 kB of archives.
After this operation, 17.8 MB of additional disk space will be used.

So, if I do the upgrade to Noble (24.04), then manually install arp-scan, then do the pialert upgrade, I don't get the errors surrounding:

*** PLEASE WAIT A COUPLE OF MINUTES...
/home/modem7/pialert/back/update_vendors.sh: 20: cd: can't cd to /usr/share/ieee-data/
sudo: get-oui: command not found

and 

Updating vendors DB...
Traceback (most recent call last):
  File "/home/modem7//pialert/back/pialert.py", line 3609, in <module>
    sys.exit(main())
             ^^^^^^

Resolution: Add arp-scan to the update (and installation) script.

leiweibau commented 2 months ago

I have now found some time. Since reacting to an operating system update within my update script becomes a bit complex (there are not only Ubuntu servers), I will make a separate “Fix_Installation” script.

What I have found out thanks to your support is:

Because of the warning (InsecureRequestWarning) These warnings are actually deactivated in the script. The assumption is that they come from the python module used for the Unifi import.

EDIT:

I will also add "arp-scan" to the update script

leiweibau commented 1 month ago

Update released with https://github.com/leiweibau/Pi.Alert/commit/afa775a1e4a14e13a1d772627af3e35f022e0461