Open jokob-sk opened 1 month ago
Hey @FlyingToto,
Let's continue the discussion here.
I've prepared a skeleton for you, here is how you load the plugin:
sudo docker-compose --env-file ../.env
in the NetAlertX location with your provided .env
file (I keep it in the parent directory). front/plugins/omada_sdn_imp/omada_sdn.py
you will find a rough boiler plate.
front/plugins/nmap_dev_scan/nmap_dev.py
I already added tplink-omada-client
into the image, so it should be available via an import
statement
Hope this gives you a good starting point. Ask away if you need help. Once the data is collected/processed correctly, it should show up here:
I've added more system columns, but they are not yet visible (SItename, SSID), and not sure if I didn't break something, but you should be OK working on the files in the front/plugins/omada_sdn_imp/
independently.
To answer your previous questions:
- can I use the device PORT field to store the SSID string name ? -> omada nicely provides the port for each switch that a device is connected to and I netalertX does allow me to track the port number per device. But for Access points, omada provides the SSID of the AP that the device is connected to...
2 since I can populate the switch/AP name per device and the port/SSID, will that build the network graph automatically for me? (that would be so cool! I better buy the older tplink switches omada compatible on ebay before people learn about that!)
- I cant remember if someone already asked, but any places to track VLANs in netalertX? or maybe even to allow custom fields? (for instance omada's api provide more details like PoE and/or network speed etc...) here is an example:
ok I will work on it on monday. 1 ) why don't you want to store the SSID name in the existing port field? (just curious) 2) " with your provided .env file" so basically this will allow me to run a second docker instance of netlartx from the new folder I pulled correct? I can just do a git pull in a new directoy, change the port and the origin locations of the /app/* mounts in the docker-compose file and run this "omada" version in parallel to my existing one? correct?
Thanks, looking forward!
ALWAYS_FRESH_INSTALL
env variable + watchtower). These instances have different docker-compose.yml
and .env
files, but your setup might differ. My dev instance uses:
docker-compose.yml
file.env
file that I supply to the docker-compose
command ( sudo docker-compose --env-file path_to_custom/.env
) that overrides the variables ${DEV_LOCATION}
, ${APP_DATA_LOCATION}
, ${TZ}
, ${PORT}
Hope this helps.
The more I think about it the more it makes sense to store SSID in the port field...
The more I think about it the more it makes sense to store SSID in the port field...
one thing a bit different is that on a switch port you only have 1 device, but on an Access Point SSID you can have multiple....
Hummm, just one FYI, the port is stored on the device that connects to that port, so if you store the SSID it's stored on the Device that connects to the SSID, not the router/AP. Maybe storing the current SSID of the device is sufficient. ANyway, those are implementation details and we can fine tune it once the main import is working :)
so... I tried to bring up my second instance but docker-compose is not happy because I already have another instance under the same name (I did change the port numbers and folder structures). is there a way to specify a different name?
this is the actual alert:
so... I tried to bring up my second instance but docker-compose is not happy because I already have another instance under the same name (I did change the port numbers and folder structures). is there a way to specify a different name?
this is the actual alert:
ok I figured it out, I updated the docker-compose.yml file to container_name: netalertxdev
now I got my second instance running.... will try to add my code to it... I
@FlyingToto great job!
I have the following setup:
Download the code:
cd /development && git clone https://github.com/jokob-sk/NetAlertX.git
touch /development/.env_dev && sudo nano /development/.env_dev
#--------------------------------
#NETALERTX
#--------------------------------
TZ=Europe/Berlin
PORT=22222
DEV_LOCATION=/development/NetAlertX
APP_DATA_LOCATION=/volume/docker_appdata
APP_CONFIG_LOCATION=/volume/
I create a folder netalertx
in the APP_DATA_LOCATION
with 2 subfolders db
and config
.
Lastly I run the container:
cd /development/NetAlertX && sudo docker-compose --env-file ../.env_dev
You can then modify the python script without restarting/rebuilding the container every time. Additionally, you can run the script via the UI:
yep all good now... just trying to figure out the python calls...
Just FYI, maybe you'll find this useful:
A command to stop, remove the container and the image (replace netalertx
and netalertx-netalertx
with the appropriate values)
sudo docker container stop netalertx ; sudo docker container rm netalertx ; sudo docker image rm netalertx-netalertx
SSH into the container and kill & restart the main script loop
sudo docker exec -it netalertx /bin/bash
pkill -f "python /app/server" && python /app/server &
Would it be possible to setup a zoom/discord call? (I am in US timezone, usually I can find time during my afternoons
I am currently able to:
I need a bit of help:
to understand the bit about the json table format that you use.
From a logical point of view, I think what I need to do is: 4.a. Parse each odevice (Omada Device) one by one. 4.b. Check if the MAC is missing from NetAlertX in which case I should create the Ndevice (NetAlertX Device) 4.c. Update the Ndevice matching the odevice MAC with the fields from omada
Example of my current tokens:
10:24:04 [<unique_prefix>] token: "['C2-52-E7-F3-A1-1E', '192.168.0.196', 'Pixel-8', 'pantry12', '(48)']"
10:24:04 [<unique_prefix>] token: "['00-E2-59-00-A0-8E', '192.168.0.1', 'bastion', 'office24', '(23)']"
10:24:04 [<unique_prefix>] token: "['E6-C6-42-C2-C7-07', '192.168.0.189', 'E6-C6-42-C2-C7-07', 'froggies3', '(ompapaoffice)']"
4.c.i. Field1 MAC -> no updates since it is used for the lookup.
4.c.ii. Field2 IP -> overwrite (I assume omada would always know better, however what if someone runs multiple IPs on the same MAC? )
4.c.iii. Field3 Hostname -> overwrite unless O_Hostname matches MAC and existing Ndevice_Hostname is not blank
4.c.iv. Field5 4.c.iv.1. If Field5 matches (number) -> overwrite Ndevice_Port_number & Field4_is_SwitchName 4.c.iv.2. Else -> overwrite Ndevice_Parent_Network_Node_MAC & Field4_is_SSID
4.c.v. Field4 4.c.v.1. For Field4_is_SwitchName -> overwrite Ndevice_Parent_Network_Node_MAC 4.c.v.2. For Field4_is_SSID -> overwrite Ndevice_SSID
Sounds like good progress. Happy to chat, can you send an email to jokob.sk@gmail.com and we can set something up 👍
In short however, you don't have to worry about when to overwrite device info. If you map the fields in this step, the application automatically does the rest.
Currently most of the fields are overwritten with new information, as far as I remember. Exceptions include Device name and I think vendor. Everything else is overwritten if a value is passed from a plugin.
Here is the code responsible for updating devices (I don't think you need to modify this part, just as a FYI)
ok so it is moving a bit....
tokens: "['CC-50-E3-00-01-02', '192.168.0.203', 'alicePC', 'mySSID', '(myaccesspoint)']"
tokens_reordered: "['CC-50-E3-00-01-02', '192.168.0.203', 'alicePC', '40-AE-30-06-07-08', 'mySSID']"
1. should we store the SSID in watched4? or just reuse watched3?
watched2 = device['some_id'], # PARENT NETWORK NODE MAC
watched3 = device['some_id'], # PORT
watched4 = device['some_id'], # SSID
3. when an omada-controlled swtich is connected to another non-omada switch itself connecte to multiple devices, they all show up under the same port. I assume it won't break anything even though you would have multiple devices all connected to a single port? example below, 3 clients all appeared to be connected to a single port=48 on switch=pantry12 because one of them is a switch not controlled by omada:
CC-50-E3-00-01-02 192.168.0.203 alicePC mySSID (myaccesspoint)
3E-3E-42-00-01-03 192.168.0.186 3E-3E-42-00-01-03 pantry12 (48)
C4-6E-7B-00-01-04 192.168.0.160 C4-6E-7B-00-01-04 pantry12 (48)
E0-3F-49-00-01-05 192.168.0.32 BobPC pantry12 (48)
Great to hear that! :)
hum... so it is not quite working yet...
1) not sure what extra and foreingKey were exactly, I dont see them in the config.json, so I just added the MAC address in foreignKey and the omada_site parameter in extra.
2) I am able to parse my devices properly (ie 1 device = "['18-C0-4D-06-F2-3C', '192.168.0.172', 'giga', 'D8-07-B6-71-FF-7F', '10']") however, once it calls plugin_objects.add_object(), it looks like each field is added separately. so I end up with a huge number of "devices" which only have 1 field populated... should add.object() take 1 list instead of 8 fields?
note: will email you the logs , script and last_result directly...
here is the main loop code:
if len(device_data) > 0:
# insert devices into the lats_result.log
# make sure the below mapping is mapped in config.json, for example:
#"database_column_definitions": [
# {
# "column": "Object_PrimaryID", <--------- the value I save into primaryId
# "mapped_to_column": "cur_MAC", <--------- gets unserted into the CurrentScan DB table column cur_MAC
# watched1 = 'null' ,
# figure a way to run my udpate script delayed
for device in device_data:
mylog('verbose', [f'[{pluginName}] main parsing device: "{device}"'])
if device[PORT_SSID].isdigit():
myport = device[PORT_SSID]
myssid = 'null'
else:
myssid = device[PORT_SSID]
myport = 'null'
plugin_objects.add_object(
primaryId = device[MAC], # MAC
secondaryId = device[IP], # IP
watched1 = device[NAME], # NAME/HOSTNAME
watched2 = device[SWITCH_AP], # PARENT NETWORK NODE MAC
watched3 = myport, # PORT
watched4 = myssid, # SSID
extra = omada_site, # SITENAME (cur_NetworkSite) or VENDOR (cur_Vendor) (PICK one and adjust config.json -> "column": "Extra")
foreignKey = device[MAC]) # usually MAC
mylog('verbose', [f'[{pluginName}] New entries: "{len(device_data)}"'])
# log result
plugin_objects.write_result_file()
Good afternoon, Any "easy" way to integrate with omada SDN? this is an example of data that we can pull from omada:
omada also has the notion of "sites" which basically divides networks into multiple locations like and the scan is on a per-site basis.
1) install python: sudo apt install python3-pip 2) install the omada python api: pip install tplink-omada-client 3) update the parameters CHANGE_ME values in the scipt below omada_username="CHANGEME_mrpotatoe" omada_password="CHANGEME_lfdaruiRWGFD335qw324z" omada_site="CHANGEME_homesweethome" omada_url="https://CHANGEME_omada.mylocaldomain" omada -t myomada target --url $omada_url --user $omada_username --password $omada_password --site $omada_site --set-default omada clients
Originally posted by @FlyingToto in https://github.com/jokob-sk/NetAlertX/discussions/707