search

Vadims06 / isiswatcher

History of changes in IS-IS domain
Other
4 stars 0 forks source link

Watcher issue #2

Open bleuchtang opened 4 days ago

bleuchtang commented 4 days ago

Hi,

I try to setup an isiswatcher but I ran into some issues with the watcher and the filter-xdp docker; They keep restarting over and over:

root@topolograph:/opt/isiswatcher# docker ps
CONTAINER ID   IMAGE                               COMMAND                  CREATED         STATUS                          PORTS                                                 NAMES
7b57edd422bd   vadims06/isis-watcher:latest        "python pysniff.py"      7 minutes ago   Up 5 seconds                                                                          watcher1-gre1-isis-watcher
d7759797b168   vadims06/isis-filter-xdp:latest     "./apply_xdp_filter.…"   8 minutes ago   Restarting (0) 15 seconds ago                                                         watcher1-gre1-receive_only_filter
d8699729cd39   vadims06/frr:v8.5.4_isis_over_gre   "/sbin/tini -- /usr/…"   8 minutes ago   Up 8 minutes                                                                          watcher1-gre1-router
[...]

From FRR everything is good; ISIS database looks OK and all my neighbors are available.

However the logs from watcher docker show:

(!) root@topolograph:/opt/isiswatcher# docker logs watcher1-gre1-isis-watcher                                                                                                               
/usr/local/lib/python3.11/site-packages/scapy/layers/ipsec.py:512: CryptographyDeprecationWarning: TripleDES has been moved to cryptography.hazmat.decrepit.ciphers.algorithms.TripleDES and 
will be removed from this module in 48.0.0.                                                                                                                                                  
  cipher=algorithms.TripleDES,                                                                                                                                                               
/usr/local/lib/python3.11/site-packages/scapy/layers/ipsec.py:516: CryptographyDeprecationWarning: TripleDES has been moved to cryptography.hazmat.decrepit.ciphers.algorithms.TripleDES and 
will be removed from this module in 48.0.0.                                                                                                                                                  
  cipher=algorithms.TripleDES,                                                                                                                                                               
lsdb_output:Area watcher1:                                                                                                                                                                   
IS-IS Level-1 link-state database:                                                            
LSP ID                  PduLen  SeqNumber   Chksum  Holdtime  ATT/P/OL                                                                                                                       
REDACTED *     43   0x00000019  0xff7e    1171    0/0/0                                                                                                                          
  Area Addr                                                                                   
ISIS LSDB has been received                                                                                                                                                                  
Sniffing packets on interface: no interface is set                                                                                                                                           
IS-IS watcher encounted the following error during sniffing: Traceback (most recent call last):                                                                                              
  File "/home/watcher/watcher/pysniff.py", line 22, in start_sniffing                                                                                                                        
    isis_sniffer.start_sniffing()                                                                                                                                                            
  File "pysniff_helper.pyx", line 255, in pysniff_helper.ISIS_INTERFACE_SNIFFER.start_sniffing                                                                                               
  File "/usr/local/lib/python3.11/site-packages/scapy/sendrecv.py", line 1311, in sniff                                                                                                      
    sniffer._run(*args, **kwargs)                                                                                                                                                            
  File "/usr/local/lib/python3.11/site-packages/scapy/sendrecv.py", line 1171, in _run                                                                                                       
    sniff_sockets[_RL2(iface)(type=ETH_P_ALL, iface=iface,                                                                                                                                   
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^                                                                                                                                     File "/usr/local/lib/python3.11/site-packages/scapy/arch/linux.py", line 495, in __init__                                                                                                  
    attach_filter(self.ins, filter, self.iface)                                               
  File "/usr/local/lib/python3.11/site-packages/scapy/arch/linux.py", line 164, in attach_filter                                                                                                 bp = compile_filter(bpf_filter, iface)                                                                                                                                                   
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^                                                                                                                                                   
  File "/usr/local/lib/python3.11/site-packages/scapy/arch/common.py", line 110, in compile_filter                                                                                           
    raise OSError(error)                                                                                                                                                                     
OSError: no interface is set: No such device exists (No such device exists)
[...]

Also, I try to find the source code of the isis-watcher docker without success, is it available somewhere ?

Thanks !

Vadims06 commented 2 days ago

Hi @bleuchtang sorry, I updated IS-IS watcher, but the documentation left non-updated (as usual). I've just released v2.0.10, please remove watcher1 folder and re-generate config

sudo docker run -it --rm --user $UID -v ./:/home/watcher/watcher/ -v /etc/passwd:/etc/passwd:ro -v /etc/group:/etc/group:ro vadims06/isis-watcher:latest python3 ./client.py --action add_watcher

Let me know if it works

bleuchtang commented 2 days ago

Thanks for update !

Everything is ok now; events are stored in mongo:

> db.adj_change.find({}).sort({_id: -1}).limit(2)
{ "_id" : ObjectId("672200d5d0726ff62c000018"), "watcher_time_iso8601" : ISODate("2024-10-30T09:48:01Z"), "event_detected_by" : "REDACTED", "graph_time" : "", "@timestamp" : ISODate("2024-10-30T09:48:02.274Z"), "old_cost" : "100", "path" : "/home/watcher/watcher/logs/watcher1-gre1-isis.isis.log", "asn" : "65444", "event_object" : "REDACTED", "watcher_name" : "isis", "event_name" : "metric", "watcher_time" : "2024-10-30T09:48:01Z", "new_cost" : "-1", "host" : "6e865c490af6", "level_number" : "1", "event_status" : "changed", "area_num" : "49.0001" }
{ "_id" : ObjectId("672200d5d0726fcb5c000017"), "watcher_time_iso8601" : ISODate("2024-10-30T09:48:01Z"), "event_detected_by" : "REDACTED", "graph_time" : "", "@timestamp" : ISODate("2024-10-30T09:48:02.275Z"), "old_cost" : "-1", "path" : "/home/watcher/watcher/logs/watcher1-gre1-isis.isis.log", "asn" : "65444", "event_object" : "REDACTED", "watcher_name" : "isis", "event_name" : "metric", "watcher_time" : "2024-10-30T09:48:01Z", "new_cost" : "100", "host" : "6e865c490af6", "level_number" : "1", "event_status" : "changed", "area_num" : "49.0001" }

But topolograph web interface is still blind, and no graph can be choose :(

Vadims06 commented 1 day ago

Hi Emile, I've noticed that graph time is empty "graph_time" : "" it means that the Watcher was not able to connect to Topolograph during its init. Could you please check that topolograph is running and you provided topolograph URL correctly in env file. Keep me posted

bleuchtang commented 1 day ago

Topolograph is running, and can be join by the watcher:

# docker exec -it watcher1-gre1-isis-isis-watcher curl -I HOST_IP:8080
HTTP/1.1 200 OK
Server: nginx/1.27.2
Date: Thu, 31 Oct 2024 09:46:07 GMT
Content-Type: text/html; charset=utf-8
Content-Length: 29420
Connection: keep-alive
Vary: Cookie
Set-Cookie: session=eyJfZnJlc2giOmZhbHNlLCJfcGVybWFuZW50Ijp0cnVlLCJhbm9ueW1vdXNfc2Vzc2lvbl9pZCI6IjY3MjM1MWRmNWZiZTljZGNlYWIxMzlhYSJ9.ZyNR3w.wFXjVBtlnh3gmwKuM3dyUsbeqrM; Expires=Sun, 01-Dec-2024 09:46:07 GMT; HttpOnly; Path=/

However I'm not sure about the credentials and the AUTHORISED_NETWORKS parts...

Is there a way to increase logs in the flask app ? I put APP_DEBUG=true in env file but It doesn't do so much.

bleuchtang commented 1 day ago

It's indeed a credential issue, when the watcher restart (because of #3), it get a 401 and can't log into the api:

# docker logs -f webserver
[...]
172.19.0.1 - topolograph@exemple.net [31/Oct/2024:12:17:18 +0000] "POST /api/watcher HTTP/1.1" 401 123 "-" "python-requests/2.32.3" 
[...]

but TOPOLOGRAPH_WEB_API_USERNAME_EMAIL and TOPOLOGRAPH_WEB_API_PASSWORD are identical between env files...