jokob-sk / NetAlertX

🖧🔍 WIFI / LAN intruder detector. Scans for devices connected to your network and alerts you if new and unknown devices are found.
GNU General Public License v3.0
2.92k stars 174 forks source link

Unifi Importer #508

Closed L0rz closed 9 months ago

L0rz commented 10 months ago

Describe the issue

When submitting an issue ❗enable debug❗ and have a look at the docs

After enable unifi importer, teh device list was broken. in the Websitelog shows this:

Uncaught Error: Invalid IP address format at formatIPlong (pialert_common.js:520:13) at devices.php:750:17 at Array.map () at Object.success (devices.php:735:28) at c (jquery.min.js:2:27742) at Object.fireWith [as resolveWith] (jquery.min.js:2:28487) at l (jquery.min.js:2:78789) at XMLHttpRequest. (jquery.min.js:2:81117)

pialert.conf:

# UNFIMP
#---------------------------
UNFIMP_RUN='schedule'
UNFIMP_RUN__metadata="{\"default_value\": \"disabled\", \"description\": [{\"language_code\": \"en_us\", \"string\": \"Enable import of devices from a UNIFI controller. If you select <code>schedule</code> the scheduling settings from below are applied. If you select <code>once</code> the scan is run only once on start of the application (container) or after you update your settings.\"}, {\"language_code\": \"es_es\", \"string\": \"Habilite la importaci\\u00f3n de dispositivos desde un controlador UNIFI. Si selecciona <code>schedule</code>, se aplican las configuraciones de programaci\\u00f3n de abajo. Si selecciona <code>once</code>, el an\\u00e1lisis se ejecuta solo una vez al iniciar la aplicaci\\u00f3n (contenedor) o despu\\u00e9s de actualizar su configuraci\\u00f3n.\"}, {\"language_code\": \"de_de\", \"string\": \"Aktiviere den Import von einem UNIFI controller. Bei <code>schedule</code> werden die Richtzeiten von weiter unten verwendet. Bei der Auswahl <code>once</code> der Import wird einmalig beim Start der Applikation (container), oder nach einem Update der Einstellungen durchgef\\u00fchrt.\"}], \"function\": \"RUN\", \"events\": [\"run\"], \"localized\": [\"name\", \"description\"], \"name\": [{\"language_code\": \"en_us\", \"string\": \"When to run\"}, {\"language_code\": \"es_es\", \"string\": \"Cu\\u00e1ndo ejecutar\"}, {\"language_code\": \"de_de\", \"string\": \"Ausf\\u00fchrungszeitpunkt\"}], \"options\": [\"disabled\", \"once\", \"schedule\", \"always_after_scan\", \"on_new_device\"], \"type\": \"text.select\"}"
UNFIMP_CMD='python3 /home/pi/pialert/front/plugins/unifi_import/script.py username={username} password={password}  host={host} sites={sites} port={port} verifyssl={verifyssl} version={version} fullimport={fullimport}'
UNFIMP_CMD__metadata="{\"default_value\": \"python3 /home/pi/pialert/front/plugins/unifi_import/script.py username={username} password={password}  host={host} sites={sites} port={port} verifyssl={verifyssl} version={version} fullimport={fullimport}\", \"description\": [{\"language_code\": \"en_us\", \"string\": \"Command to run. Not recommended to change.\"}, {\"language_code\": \"es_es\", \"string\": \"Comando para ejecutar. No se recomienda cambiar.\"}, {\"language_code\": \"de_de\", \"string\": \"Befehl der ausgef\\u00fchrt wird. Anpassung nicht empfohlen\"}], \"function\": \"CMD\", \"localized\": [\"name\", \"description\"], \"name\": [{\"language_code\": \"en_us\", \"string\": \"Command\"}, {\"language_code\": \"es_es\", \"string\": \"Comando\"}, {\"language_code\": \"de_de\", \"string\": \"Befehl\"}], \"options\": [], \"type\": \"text\"}"
UNFIMP_username='xxxxx'
UNFIMP_username__metadata="{\"default_value\": \"\", \"description\": [{\"language_code\": \"en_us\", \"string\": \"The username used to login into your UNIFI controller. It is recommended to create a read-only user account.\"}, {\"language_code\": \"es_es\", \"string\": \"El nombre de usuario utilizado para iniciar sesi\\u00f3n en su controlador UNIFI. Se recomienda crear una cuenta de usuario de s\\u00f3lo lectura.\"}], \"function\": \"username\", \"localized\": [\"name\", \"description\"], \"name\": [{\"language_code\": \"en_us\", \"string\": \"Username\"}, {\"language_code\": \"es_es\", \"string\": \"Nombre de usuario\"}], \"options\": [], \"type\": \"text\"}"
UNFIMP_password='xxxxxx'
UNFIMP_password__metadata="{\"default_value\": \"\", \"description\": [{\"language_code\": \"en_us\", \"string\": \"The password used to login into your UNIFI controller.\"}, {\"language_code\": \"es_es\", \"string\": \"La contrase\\u00f1a utilizada para iniciar sesi\\u00f3n en su controlador UNIFI.\"}], \"function\": \"password\", \"localized\": [\"name\", \"description\"], \"name\": [{\"language_code\": \"en_us\", \"string\": \"Password\"}, {\"language_code\": \"es_es\", \"string\": \"Contrase\\u00f1a\"}], \"options\": [], \"type\": \"password\"}"
UNFIMP_host='192.168.130.10'
UNFIMP_host__metadata="{\"default_value\": \"192.168.1.1\", \"description\": [{\"language_code\": \"en_us\", \"string\": \"The host (IP) where the UNIFI controller is runnig. Do NOT include the protocol (e.g. <code>https://</code>)\"}, {\"language_code\": \"es_es\", \"string\": \"El host (IP) donde se ejecuta el controlador UNIFI. NO incluya el protocolo (por ejemplo, <code>https://</code>)\"}], \"function\": \"host\", \"localized\": [\"name\", \"description\"], \"name\": [{\"language_code\": \"en_us\", \"string\": \"Host\"}, {\"language_code\": \"es_es\", \"string\": \"Host\"}], \"options\": [], \"type\": \"text\"}"
UNFIMP_port='8443'
UNFIMP_port__metadata="{\"default_value\": \"8443\", \"description\": [{\"language_code\": \"en_us\", \"string\": \"The port number where the UNIFI controller is runnig. Usually it is <code>8443</code>, for UDM(P) devices its 443.\"}, {\"language_code\": \"es_es\", \"string\": \"El n\\u00famero de puerto donde se ejecuta el controlador UNIFI. Normalmente es <code>8443</code>.\"}], \"function\": \"port\", \"localized\": [\"name\", \"description\"], \"name\": [{\"language_code\": \"en_us\", \"string\": \"Port number\"}, {\"language_code\": \"es_es\", \"string\": \"N\\u00famero de puerto\"}], \"options\": [], \"type\": \"text\"}"
UNFIMP_verifyssl='false'
UNFIMP_verifyssl__metadata="{\"default_value\": \"false\", \"description\": [{\"language_code\": \"en_us\", \"string\": \"verify SSL certificate validity <code>true|false</code>.\"}, {\"language_code\": \"es_es\", \"string\": \"verificar la validez del certificado SSL <code>true|false</code>.\"}], \"function\": \"verifyssl\", \"localized\": [\"name\", \"description\"], \"name\": [{\"language_code\": \"en_us\", \"string\": \"verify SSL\"}, {\"language_code\": \"es_es\", \"string\": \"verificar SSL\"}], \"options\": [\"true\", \"false\"], \"type\": \"text.select\"}"
UNFIMP_version='v5'
UNFIMP_version__metadata="{\"function\": \"version\", \"type\": \"text\", \"default_value\": \"v4\", \"options\": [], \"localized\": [\"name\", \"description\"], \"name\": [{\"language_code\": \"en_us\", \"string\": \"API version\"}, {\"language_code\": \"es_es\", \"string\": \"Versi\\u00f3n API\"}], \"description\": [{\"language_code\": \"en_us\", \"string\": \"The base version of the Unify controller API. Supported values as of time of writing are <code>v4|v5|unifiOS|UDMP-unifiOS</code>.\"}, {\"language_code\": \"es_es\", \"string\": \"La versi\\u00f3n base de la API del controlador Unify. Los valores admitidos al momento de escribir este art\\u00edculo son <code>v4|v5|unifiOS|UDMP-unifiOS</code>.\"}]}"
UNFIMP_sites=['default']
UNFIMP_sites__metadata="{\"default_value\": [\"default\"], \"description\": [{\"language_code\": \"en_us\", \"string\": \"The sites you want to connect to. Usually it is only one and the name is <code>default</code>. Check the URL in your UniFi controller UI if unsure.\"}, {\"language_code\": \"es_es\", \"string\": \"Los sitios a los que desea conectarse. Generalmente es solo uno y el nombre es <code>default</code>. Verifique la URL en la interfaz de usuario de su controlador UniFi si no est\\u00e1 seguro.\"}], \"function\": \"sites\", \"localized\": [\"name\", \"description\"], \"name\": [{\"language_code\": \"en_us\", \"string\": \"UNIFI sites\"}, {\"language_code\": \"es_es\", \"string\": \"Sitios UNIFI\"}], \"options\": [], \"type\": \"list\"}"
UNFIMP_RUN_SCHD='*/5 * * * *'
UNFIMP_RUN_SCHD__metadata="{\"default_value\": \"0 2 * * *\", \"description\": [{\"language_code\": \"en_us\", \"string\": \"Only enabled if you select <code>schedule</code> in the <a href=\\\"#UNFIMP_RUN\\\"><code>UNFIMP_RUN</code> setting</a>. Make sure you enter the schedule in the correct cron-like format (e.g. validate at <a href=\\\"https://crontab.guru/\\\" target=\\\"_blank\\\">crontab.guru</a>). For example entering <code>0 4 * * *</code> will run the scan after 4 am in the <a onclick=\\\"toggleAllSettings()\\\" href=\\\"#TIMEZONE\\\"><code>TIMEZONE</code> you set above</a>. Will be run NEXT time the time passes. <br/> It's recommended to use the same schedule interval for all plugins responsible for discovering new devices. \"}, {\"language_code\": \"es_es\", \"string\": \"Solo est\\u00e1 habilitado si selecciona <code>schedule</code> en la configuraci\\u00f3n <a href=\\\"#UNFIMP_RUN\\\"><code>UNFIMP_RUN</code></a>. Aseg\\u00farese de ingresar la programaci\\u00f3n en el formato similar a cron correcto (por ejemplo, valide en <a href=\\\"https://crontab.guru/\\\" target=\\\"_blank\\\">crontab.guru</a>). Por ejemplo, ingresar <code>0 4 * * *</code> ejecutar\\u00e1 el escaneo despu\\u00e9s de las 4 a.m. en el <a onclick=\\\"toggleAllSettings()\\\" href=\\\"#TIMEZONE\\\"><code>TIMEZONE</ c\\u00f3digo> que configur\\u00f3 arriba</a>. Se ejecutar\\u00e1 la PR\\u00d3XIMA vez que pase el tiempo.\"}], \"function\": \"RUN_SCHD\", \"localized\": [\"name\", \"description\"], \"name\": [{\"language_code\": \"en_us\", \"string\": \"Schedule\"}, {\"language_code\": \"es_es\", \"string\": \"Schedule\"}], \"options\": [], \"type\": \"text\"}"
UNFIMP_RUN_TIMEOUT=5
UNFIMP_RUN_TIMEOUT__metadata="{\"default_value\": 5, \"description\": [{\"language_code\": \"en_us\", \"string\": \"Maximum time in seconds to wait for the script to finish. If this time is exceeded the script is aborted.\"}, {\"language_code\": \"es_es\", \"string\": \"Tiempo m\\u00e1ximo en segundos para esperar a que finalice el script. Si se supera este tiempo, el script se cancela.\"}, {\"language_code\": \"de_de\", \"string\": \"Maximale Laufzeit des Scripts. Nach Ablauf der Zeit wird das Script abgebrochen.\"}], \"function\": \"RUN_TIMEOUT\", \"localized\": [\"name\", \"description\"], \"name\": [{\"language_code\": \"en_us\", \"string\": \"Run timeout\"}, {\"language_code\": \"es_es\", \"string\": \"Tiempo l\\u00edmite de ejecuci\\u00f3n\"}, {\"language_code\": \"de_de\", \"string\": \"Wartezeit\"}], \"options\": [], \"type\": \"integer\"}"
UNFIMP_WATCH=['Watched_Value1']
UNFIMP_WATCH__metadata="{\"default_value\": [\"Watched_Value1\", \"Watched_Value4\"], \"description\": [{\"language_code\": \"en_us\", \"string\": \"Send a notification if selected values change. Use <code>CTRL + Click</code> to select/deselect. <ul> <li><code>Watched_Value1</code> is Hostname </li><li><code>Watched_Value2</code> is Vendor </li><li><code>Watched_Value3</code> is Type </li><li><code>Watched_Value4</code> is Online </li></ul>\"}, {\"language_code\": \"es_es\", \"string\": \"Env\\u00ede una notificaci\\u00f3n si los valores seleccionados cambian. Utilice <code>CTRL + clic</code> para seleccionar/deseleccionar. <ul> <li><code>Watched_Value1</code> es el nombre de host </li><li><code>Watched_Value2</code> es el proveedor </li><li><code>Watched_Value3</code> es el tipo </li><li><code>Watched_Value4</code> es Online </li></ul>\"}], \"function\": \"WATCH\", \"localized\": [\"name\", \"description\"], \"name\": [{\"language_code\": \"en_us\", \"string\": \"Watched\"}, {\"language_code\": \"es_es\", \"string\": \"Visto\"}], \"options\": [\"Watched_Value1\", \"Watched_Value2\", \"Watched_Value3\", \"Watched_Value4\"], \"type\": \"text.multiselect\"}"
UNFIMP_REPORT_ON=['new','watched-changed']
UNFIMP_REPORT_ON__metadata="{\"default_value\": [\"new\", \"watched-changed\"], \"description\": [{\"language_code\": \"en_us\", \"string\": \"Send a notification only on these statuses. <code>new</code> means a new unique (unique combination of PrimaryId and SecondaryId) object was discovered. <code>watched-changed</code> means that selected <code>Watched_ValueN</code> columns changed.\"}, {\"language_code\": \"es_es\", \"string\": \"Env\\u00ede una notificaci\\u00f3n solo en estos estados. <code>new</code> significa que se descubri\\u00f3 un nuevo objeto \\u00fanico (una combinaci\\u00f3n \\u00fanica de PrimaryId y SecondaryId). <code>watched-changed</code> significa que las columnas <code>Watched_ValueN</code> seleccionadas cambiaron.\"}], \"function\": \"REPORT_ON\", \"localized\": [\"name\", \"description\"], \"name\": [{\"language_code\": \"en_us\", \"string\": \"Report on\"}, {\"language_code\": \"es_es\", \"string\": \"Informar sobre\"}], \"options\": [\"new\", \"watched-changed\", \"watched-not-changed\", \"missing-in-last-scan\"], \"type\": \"text.multiselect\"}"
UNFIMP_fullimport='once'
UNFIMP_fullimport__metadata="{\"default_value\": \"disabled\", \"description\": [{\"language_code\": \"en_us\", \"string\": \"Enable a full import of all known devices by the controller instead of only the online ones. During the scan, all found devices appear online. If you select <code>once</code> the full import is only performed once and the status is set to <code>done</code> afterwards.\"}, {\"language_code\": \"es_es\", \"string\": \"Translation Needed - Enable a full import of all known devices by the controller instead of only the online ones. During the scan, all found devices appear online. If you select <code>once</code> the full import is only performed once and the status is set to <code>done</code> afterwards.\"}, {\"language_code\": \"de_de\", \"string\": \"F\\u00fchre einen kompletten Import aller dem Controller bekannten Ger\\u00e4ten durch. W\\u00e4hrend des Scans werden alle gefundenen Ger\\u00e4te einmalig als Online angezeigt. Bei der Auswahl von <code>once</code> wird der Scan einmalig durchgef\\u00fchrt und der Status anschliessend auf <code>done</code> gesetzt\"}], \"function\": \"fullimport\", \"localized\": [\"name\", \"description\"], \"name\": [{\"language_code\": \"en_us\", \"string\": \"Perform full import\"}, {\"language_code\": \"es_es\", \"string\": \"[ES] Perform full import\"}, {\"language_code\": \"de_de\", \"string\": \"Vollst\\u00e4ndigen Import durchf\\u00fchren\"}], \"options\": [\"disabled\", \"once\", \"always\"], \"type\": \"text.select\"}"

Screenshots

image

Paste last few lines from pialert.log

11:41:11 [Plugin utils] description: This plugin is to execute an arp-scan on the local network 11:41:11 [Plugin utils] --------------------------------------------- 11:41:11 [Plugin utils] display_name: CSV backup 11:41:11 [Plugin utils] description: A plugin to auto-generate devices.csv backups. 11:41:11 [Plugin utils] --------------------------------------------- 11:41:11 [Plugin utils] display_name: DB cleanup 11:41:11 [Plugin utils] description: A plugin to schedule database cleanup & upkeep tasks. 11:41:11 [Plugin utils] --------------------------------------------- 11:41:11 [Plugin utils] display_name: DDNS update 11:41:11 [Plugin utils] description: A plugin update the DDNS record. 11:41:11 [Plugin utils] --------------------------------------------- 11:41:11 [Plugin utils] display_name: DHCP Leases (Device import) 11:41:11 [Plugin utils] description: This plugin is to import devices from dhcp.leases files. 11:41:11 [Plugin utils] --------------------------------------------- 11:41:11 [Plugin utils] display_name: Rogue DHCP 11:41:11 [Plugin utils] description: This plugin is to use NMAP to monitor for rogue DHCP servers. 11:41:11 [Plugin utils] --------------------------------------------- 11:41:11 [Plugin utils] display_name: Internet-Check 11:41:11 [Plugin utils] description: A plugin to check your internet connectivity and IP. 11:41:12 [Plugin utils] --------------------------------------------- 11:41:12 [Plugin utils] display_name: Internet speedtest 11:41:12 [Plugin utils] description: A plugin to perform a scheduled internet speedtest. 11:41:12 [Plugin utils] --------------------------------------------- 11:41:12 [Plugin utils] display_name: Maintenance 11:41:12 [Plugin utils] description: A plugin for maintenance tasks. 11:41:12 [Plugin utils] --------------------------------------------- 11:41:12 [Plugin utils] display_name: New Devices 11:41:12 [Plugin utils] description: The template used for new devices. 11:41:12 [Plugin utils] --------------------------------------------- 11:41:12 [Plugin utils] display_name: Services & Ports (NMAP) 11:41:12 [Plugin utils] description: This plugin shows all services discovered by NMAP scans. 11:41:12 [Plugin utils] --------------------------------------------- 11:41:12 [Plugin utils] display_name: Pholus-Scan (Name discovery) 11:41:12 [Plugin utils] description: This plugin is to execute a Pholus-scan (name discovery) on the local network 11:41:12 [Plugin utils] --------------------------------------------- 11:41:12 [Plugin utils] display_name: PiHole (Device sync) 11:41:12 [Plugin utils] description: This plugin syncs devices from the PiHole database 11:41:12 [Plugin utils] --------------------------------------------- 11:41:12 [Plugin utils] display_name: Set password 11:41:12 [Plugin utils] description: A simple plugin to set the web ui password on app start. 11:41:12 [Plugin utils] --------------------------------------------- 11:41:12 [Plugin utils] display_name: SNMP discovery 11:41:12 [Plugin utils] description: This plugin is used to discover devices via the arp table(s) of a RFC1213 compliant router or switch. 11:41:12 [Plugin utils] --------------------------------------------- 11:41:12 [Plugin utils] display_name: Un-Discoverable Devices 11:41:12 [Plugin utils] description: This plugin is to import undiscoverable devices from a file. 11:41:12 [Plugin utils] --------------------------------------------- 11:41:12 [Plugin utils] display_name: UniFi import 11:41:12 [Plugin utils] description: This plugin is used to import devices from an UNIFI controller. 11:41:13 [Plugin utils] --------------------------------------------- 11:41:13 [Plugin utils] display_name: Vendor update 11:41:13 [Plugin utils] description: A plugin to schedule vendor database updates for mac based vendor resolution. 11:41:13 [Plugin utils] --------------------------------------------- 11:41:13 [Plugin utils] display_name: Website monitor 11:41:13 [Plugin utils] description: This plugin is to monitor status changes of services or websites. 11:41:14 [API] Updating table_settings.json file in /front/api 11:41:14 [Config] Imported new config 11:41:14 [API] Updating table_plugins_language_strings.json file in /front/api 11:41:14 [Notification] Check if something to report 11:41:14 [Notification] Open text Template 11:41:14 [Notification] Open html Template 11:41:14 [Notification] Using template/back/report_template.html 11:41:14 [Notification] included sections: ['newdevices', 'events'] 11:41:14 [Notification] New Devices sections done. 11:41:14 [Notification] Events sections done. 11:41:14 [Notification] Udating API files 11:41:14 [Send API] Updating notification files in /home/pi/pialert/front/api/ 11:41:14 [Notification] No changes to report 11:41:14 [MAIN] Process: Wait 11:41:19 [MAIN] waiting to start next loop 11:41:24 [MAIN] waiting to start next loop 11:41:29 [MAIN] waiting to start next loop 11:41:34 [MAIN] waiting to start next loop 11:41:39 [MAIN] waiting to start next loop 11:41:44 [MAIN] waiting to start next loop 11:41:49 [MAIN] waiting to start next loop 11:41:54 [MAIN] waiting to start next loop 11:41:59 [MAIN] waiting to start next loop 11:42:04 [MAIN] waiting to start next loop 11:42:09 [MAIN] waiting to start next loop 11:42:15 [Notification] Check if something to report 11:42:15 [Notification] Open text Template 11:42:15 [Notification] Open html Template 11:42:15 [Notification] Using template/back/report_template.html 11:42:15 [Notification] included sections: ['newdevices', 'events'] 11:42:15 [Notification] New Devices sections done. 11:42:15 [Notification] Events sections done. 11:42:15 [Notification] Udating API files 11:42:15 [Send API] Updating notification files in /home/pi/pialert/front/api/ 11:42:15 [Notification] No changes to report 11:42:15 [MAIN] Process: Wait 11:42:20 [MAIN] waiting to start next loop 11:42:25 [MAIN] waiting to start next loop 11:42:30 [MAIN] waiting to start next loop 11:42:35 [MAIN] waiting to start next loop 11:42:40 [MAIN] waiting to start next loop 11:42:45 [MAIN] waiting to start next loop 11:42:50 [MAIN] waiting to start next loop 11:42:55 [MAIN] waiting to start next loop 11:43:00 [MAIN] waiting to start next loop



# paste code below 
jokob-sk commented 10 months ago

Hi there,

I just tested this on my end without encountering issues. Your names or other Unifi values may contain a special character breaking the import. Can you have a look at the imported data under PLugins to see if you can spot any potential weird looking entries?

image

Ideally also provide the section of pialert.log (after enabling debug) when the import is happening. The debug log should contain detailed data that is inserted into the DB, here is an example how this could look like:

image

This way I can see exactly what the Plugin is trying to import.

Thanks, j

L0rz commented 10 months ago

now are my Logfile and Screenshot of this. pialert.log image

Thx for help!

jokob-sk commented 10 months ago

Thanks for the details! This should be fixed in the next release - can you try checking the pi.alert_dev image in about ~20 min? The fix should be in it.

L0rz commented 10 months ago

Yes it's fixed!

jokob-sk commented 10 months ago

Great, keeping it open for other people to find it 👍

jokob-sk commented 9 months ago

closing as released 🚀