search

networktocode / ntc-templates

TextFSM templates for parsing show commands of network devices
https://ntc-templates.readthedocs.io/
Other
1.12k stars 730 forks source link

Issue: cisco_asa, show vpn-sessiondb detail l2l not parsing #1358

Closed ArxBloemJo closed 3 months ago

ArxBloemJo commented 1 year ago
ISSUE TYPE
TEMPLATE USING
Value Filldown,Required SESSION_TYPE (\S+)
Value Filldown CONNECTION (\d+\.\d+\.\d+\.\d+)
Value Filldown INDEX (\d+)
Value Filldown IP_ADDRESS (\d+\.\d+\.\d+\.\d+)
Value Filldown PROTOCOL (.+?)
Value Filldown ENCRYPTION (.+?)
Value Filldown HASHING (.+?)
Value Filldown TOTAL_BYTES_TRANSMITTED (\d+)
Value Filldown TOTAL_BYTES_RECEIVED (\d+)
Value Filldown LOGIN_TIME (\d+:\d+:\d+)
Value Filldown LOGIN_TIME_ZONE (\S+)
Value Filldown LOGIN_WEEKDAY (\w+)
Value Filldown LOGIN_MONTH (\w+)
Value Filldown LOGIN_DAY (\d+)
Value Filldown LOGIN_YEAR (\d+)
Value Filldown DURATION (.+?)
Value Filldown FILTER_NAME (.*?)
Value Filldown TOTAL_IKE_SESSIONS (\d+)
Value Filldown TOTAL_IPSEC_SESSIONS (\d+)
Value CONNECTION_TYPE (\S+)
Value SESSION_ID (\d+)
Value UDP_SRC_PORT (\d+)
Value UDP_DST_PORT (\d+)
Value NEGOTIAION_MODE (\w+)
Value AUTHENTICATION_MODE (\w+)
Value REMOTE_AUTHENTICATION_MODE (\S+|)
Value LOCAL_AUTHENTICATION_MODE (\S+|)
Value ENCRYPTION_METHOD (\S+)
Value HASH_METHOD (\w+)
Value REKEY_INTERVAL (\d+)
Value REKEY_INTERVAL_UNIT (\S+)
Value REKEY_TIME_LEFT (\d+)
Value REKEY_TIME_LEFT_UNIT (\S+)
Value REKEY_DATA_INTERVAL (\d+)
Value REKEY_DATA_INTERVAL_UNIT (\S+)
Value REKEY_DATA_REMAINING (\d+)
Value REKEY_DATA_REMAINING_UNIT (\S+)
Value IDLE_TIMEOUT_INTERVAL (\d+)
Value IDLE_TIMEOUT_INTERVAL_UNIT (\S+)
Value IDLE_TIMEOUT_REMAINING (\d+)
Value IDLE_TIMEOUT_REMAINING_UNIT (\S+)
Value PRF (\S+)
Value DH_GROUP (\d+)
Value IPV6_FILTER_NAME (.*?)
Value LOCAL_ADDRESS_NETWORK (\d+\.\d+\.\d+\.\d+)
Value LOCAL_ADDRESS_MASK (\d+\.\d+\.\d+\.\d+)
Value REMOTE_ADDRESS_NETWORK (\d+\.\d+\.\d+\.\d+)
Value REMOTE_ADDRESS_MASK (\d+\.\d+\.\d+\.\d+)
Value ENCAPSULATION (\w+)
Value PFS_GROUP (\d+)
Value BYTES_TRANSMITTED (\d+)
Value BYTES_RECEIVED (\d+)
Value PACKETS_TRANSMITTED (\d+)
Value PACKETS_RECEIVED (\d+)
Value REVAL_TIMEOUT (\d+)
Value REVAL_TIMOUT_UNIT (\S+)
Value REVAL_TIMEOUT_REMAINING (\d+)
Value REVAL_TIMEOUT_REMAINING_UNIT (\S+)
Value STATUS_QUERY_INTERVAL (\S+)
Value STATUS_QUERY_INTERVAL_UNIT (\S+)
Value EAP_OVER_UDP_TIMER (\d+)
Value EAP_OVER_UDP_TIMER_UNIT (\S+)
Value POSTURE_HOLDTIME_REMAINING (\d+)
Value POSTURE_HOLDTIME_REMAINING_UNIT (\S+)
Value POSTURE_TOKEN (.*?)
Value REDIRECT_URL (.*?)

Start
  ^Session\s+Type:\s+${SESSION_TYPE}\s+Detailed\s*$$ -> Connection

Connection
  ^\s*Connection\s*:\s+${CONNECTION}\s*$$
  ^\s*Index\s*:\s+${INDEX}\s+IP\s+Addr\s*:\s+${IP_ADDRESS}\s*$$
  ^\s*Protocol\s*:\s+${PROTOCOL}(?:\s+Encryption\s*:\s+${ENCRYPTION}|)\s*$$
  ^\s*Encryption\s*:\s+${ENCRYPTION}\s+Hashing\s*:\s+${HASHING}\s*$$
  ^\s*Encryption\s*:\s+${ENCRYPTION}\s*$$
  ^\s*Hashing\s*:\s+${HASHING}\s*$$
  ^\s*Bytes\s+Tx\s*:\s+${TOTAL_BYTES_TRANSMITTED}\s+Bytes\s+Rx\s*:\s+${TOTAL_BYTES_RECEIVED}\s*$$
  ^\s*Login\s+Time\s*:\s+${LOGIN_TIME}\s+${LOGIN_TIME_ZONE}\s+${LOGIN_WEEKDAY}\s+${LOGIN_MONTH}\s+${LOGIN_DAY}\s+${LOGIN_YEAR}\s*$$
  ^\s*Duration\s*:\s+${DURATION}\s*$$
  ^\s*Filter\s+Name\s*:\s*${FILTER_NAME}\s*$$
  ^\s*IKE(?:[Vv]\d|)\s+Sessions:\s+${TOTAL_IKE_SESSIONS}\s+IPSec\s+Sessions:\s+${TOTAL_IPSEC_SESSIONS}\s*$$
  ^\s*IKE(?:[Vv]\d|)\s+Tunnels:\s*${TOTAL_IKE_SESSIONS}\s*$$
  ^\s*IP[Ss]ec\s+Tunnels:\s*${TOTAL_IPSEC_SESSIONS}\s*$$
  ^\s*IP[Ss]ecOverNatT\s+Tunnels:\s*${TOTAL_IPSEC_SESSIONS}\s*$$
  ^\s*${CONNECTION_TYPE}:\s*$$ -> Continue
  ^\s*IKE(?:[Vv]\d|): -> IKE
  ^\s*IP[Ss]ec: -> IPSec
  ^\s*NAC: -> NAC
  ^\s*Connection\s*: -> Continue.Record
  ^\s*Connection\s*:\s+${CONNECTION}\s*$$
  ^Session\s+Type -> Continue.Record
  ^Session\s+Type -> Continue.Clearall
  ^Session\s+Type:\s+${SESSION_TYPE}\s+Detailed\s*$$
  ^\s*$$
  ^. -> Error

IKE
  ^\s*(Session|Tunnel)\s+ID\s*:\s+(?:\d+\.|)${SESSION_ID}\s*$$
  ^\s*UDP\s+Src\s+Port\s*:\s+${UDP_SRC_PORT}\s+UDP\s+Dst\s+Port\s*:\s+${UDP_DST_PORT}\s*$$
  ^\s*Rem\s+Auth\s+Mode\s*:\s*${REMOTE_AUTHENTICATION_MODE}\s*$$
  ^\s*Loc\s+Auth\s+Mode\s*:\s*${LOCAL_AUTHENTICATION_MODE}\s*$$
  ^\s*IKE\s+Neg\s+Mode\s*:\s+${NEGOTIAION_MODE}\s+Auth\s+Mode\s*:\s+${AUTHENTICATION_MODE}\s*$$
  ^\s*Encryption\s*:\s+${ENCRYPTION_METHOD}\s+Hashing\s*:\s+${HASH_METHOD}\s*$$
  ^\s*Encapsulation\s+:\s*${ENCAPSULATION}\s*$$
  ^\s*Rekey\s+Int\s+\([Tt]\):\s+${REKEY_INTERVAL}\s+${REKEY_INTERVAL_UNIT}\s+Rekey\s+Left\([Tt]\):\s+${REKEY_TIME_LEFT}\s+${REKEY_TIME_LEFT_UNIT}\s*$$
  ^\s*Rekey\s+Int\s+\([Dd]\):\s+${REKEY_DATA_INTERVAL}\s+${REKEY_DATA_INTERVAL_UNIT}\s+Rekey\s+Left\([Dd]+\):\s+${REKEY_DATA_REMAINING}\s+${REKEY_DATA_REMAINING_UNIT}\s*$$
  ^\s*(?:PRF\s*:\s+${PRF}\s+|)D\/H\s+Group\s*:\s+${DH_GROUP}\s*$$
  ^\s*Filter\s+Name\s+:\s*${FILTER_NAME}\s*$$
  ^\s*IPv6\s+Filter\s+:\s*${IPV6_FILTER_NAME}\s*$$
  ^\s*\S+:\s*$$ -> Continue.Record
  ^\s*${CONNECTION_TYPE}:\s*$$ -> Continue
  ^\s*IKE(?:[Vv]\d|): -> IKE
  ^\s*IP[Ss]ec(?:OverNatT|): -> IPSec
  ^\s*NAC: -> NAC
  ^\s*Connection\s*: -> Continue.Record
  ^\s*Connection\s*:\s+${CONNECTION}\s*$$ -> Connection
  ^Session\s+Type -> Continue.Record
  ^Session\s+Type -> Continue.Clearall
  ^Session\s+Type:\s+${SESSION_TYPE}\s+Detailed\s*$$ -> Connection
  ^\s*$$
  ^. -> Error

IPSec
  ^\s*(Session|Tunnel)\s+ID\s*:\s+(?:\d+\.|)${SESSION_ID}\s*$$
  ^\s*Local\s+Addr\s*:\s+${LOCAL_ADDRESS_NETWORK}\/${LOCAL_ADDRESS_MASK}
  ^\s*Remote\s+Addr\s*:\s+${REMOTE_ADDRESS_NETWORK}\/${REMOTE_ADDRESS_MASK}
  ^\s*Encryption\s*:\s+${ENCRYPTION_METHOD}\s+Hashing\s*:\s+${HASH_METHOD}\s*$$
  ^\s*Encapsulation\s*:\s+${ENCAPSULATION}(?:\s+PFS\s+Group\s*:\s+${PFS_GROUP}|)\s*$$
  ^\s*Rekey\s+Int\s+\([Tt]\):\s+${REKEY_INTERVAL}\s+${REKEY_INTERVAL_UNIT}\s+Rekey\s+Left\([Tt]\):\s+${REKEY_TIME_LEFT}\s+${REKEY_TIME_LEFT_UNIT}\s*$$
  ^\s*Rekey\s+Int\s+\([Dd]\):\s+${REKEY_DATA_INTERVAL}\s+${REKEY_DATA_INTERVAL_UNIT}\s+Rekey\s+Left\([Dd]+\):\s+${REKEY_DATA_REMAINING}\s+${REKEY_DATA_REMAINING_UNIT}\s*$$
  ^\s*Idle\s+Time\s+Out\s*:\s+${IDLE_TIMEOUT_INTERVAL}\s+${IDLE_TIMEOUT_INTERVAL_UNIT}\s+Idle\s+TO\s+Left\s*:\s+${IDLE_TIMEOUT_REMAINING}\s+${IDLE_TIMEOUT_REMAINING_UNIT}\s*$$             
  ^\s*Bytes\s+Tx\s*:\s+${BYTES_TRANSMITTED}\s+Bytes\s+Rx\s*:\s+${BYTES_RECEIVED}\s*$$
  ^\s*Pkts\s+Tx\s*:\s+${PACKETS_TRANSMITTED}\s+Pkts\s+Rx\s*:\s+${PACKETS_RECEIVED}\s*$$
  ^\s*\S+:\s*$$ -> Continue.Record
  ^\s*${CONNECTION_TYPE}:\s*$$ -> Continue
  ^\s*IKE(?:[Vv]\d|): -> IKE
  ^\s*IP[Ss]ec(?:OverNatT|): -> IPSec
  ^\s*NAC: -> NAC
  ^\s*Connection\s*: -> Continue.Record
  ^\s*Connection\s*:\s+${CONNECTION}\s*$$ -> Connection
  ^Session\s+Type -> Continue.Record
  ^Session\s+Type -> Continue.Clearall
  ^Session\s+Type:\s+${SESSION_TYPE}\s+Detailed\s*$$ -> Connection
  ^\s*$$
  ^. -> Error

NAC
  ^\s*Reval\s+Int\s+\(\w\)\s*:\s+${REVAL_TIMEOUT}\s+${REVAL_TIMOUT_UNIT}\s+Reval\s+Left\s*\(\w\)\s*:\s+${REVAL_TIMEOUT_REMAINING}\s+${REVAL_TIMEOUT_REMAINING_UNIT}\s*$$
  ^\s*SQ\s+Int\s+\(\w\)\s*:\s+${STATUS_QUERY_INTERVAL}\s+${STATUS_QUERY_INTERVAL_UNIT}\s+EoU\s+Age\(\w\)\s*:\s+${EAP_OVER_UDP_TIMER}\s+${EAP_OVER_UDP_TIMER_UNIT}\s*$$
  ^\s*Hold\s+Left\s+\(\w\)\s*:\s+${POSTURE_HOLDTIME_REMAINING}\s+${POSTURE_HOLDTIME_REMAINING_UNIT}\s+Posture\s+Token\s*:\s*${POSTURE_TOKEN}\s*$$
  ^\s*Redirect\s+URL\s*:\s*${REDIRECT_URL}\s*$$
  ^\s*\S+:\s*$$ -> Continue.Record
  ^\s*${CONNECTION_TYPE}:\s*$$ -> Continue
  ^\s*IKE(?:[Vv]\d|): -> IKE
  ^\s*IP[Ss]ec(?:OverNatT|): -> IPSec
  ^\s*NAC: -> NAC
  ^\s*Connection\s*: -> Continue.Record
  ^\s*Connection\s*:\s+${CONNECTION}\s*$$ -> Connection
  ^Session\s+Type -> Continue.Record
  ^Session\s+Type -> Continue.Clearall
  ^Session\s+Type:\s+${SESSION_TYPE}\s+Detailed\s*$$ -> Connection
  ^\s*$$
  ^. -> Error
SAMPLE COMMAND OUTPUT
Connection   : DefaultL2LGroup
Index        : 195354                 IP Addr      : 145.224.99.131
Protocol     : IKEv2 IPsecOverNatT
Encryption   : IKEv2: (1)AES256  IPsecOverNatT: (1)AES256
Hashing      : IKEv2: (1)SHA256  IPsecOverNatT: (1)SHA256
Bytes Tx     : 100158242              Bytes Rx     : 104530574
Login Time   : 00:13:25 CEST Thu Apr 20 2023
Duration     : 15h:26m:01s

IKEv2 Tunnels: 1
IPsecOverNatT Tunnels: 1

IKEv2:
  Tunnel ID    : 195354.1
  UDP Src Port : 43022                  UDP Dst Port : 4500
  Rem Auth Mode: preSharedKeys
  Loc Auth Mode: preSharedKeys
  Encryption   : AES256                 Hashing      : SHA256
  Rekey Int (T): 86400 Seconds          Rekey Left(T): 30854 Seconds
  PRF          : SHA256                 D/H Group    : 14
  Filter Name  :

IPsecOverNatT:
  Tunnel ID    : 195354.2
  Local Addr   : 10.3.148.0/255.255.252.0/0/0
  Remote Addr  : 10.250.0.0/255.255.0.0/0/0
  Encryption   : AES256                 Hashing      : SHA256
  Encapsulation: Tunnel                 PFS Group    : 14
  Rekey Int (T): 28800 Seconds          Rekey Left(T): 26325 Seconds
  Rekey Int (D): 4608000 K-Bytes        Rekey Left(D): 4605988 K-Bytes
  Idle Time Out: 30 Minutes             Idle TO Left : 29 Minutes
  Bytes Tx     : 100159543              Bytes Rx     : 104531656
  Pkts Tx      : 239110                 Pkts Rx      : 242980
SUMMARY
EXPECTED RESULTS

Currently parsed data

ACTUAL RESULTS
Traceback (most recent call last):
  File "c:\Users\JoeriBloemen\OneDrive - Arxus.cloud\Arxus\LocalCoding\ASA\Billing_netw\asa_billing.py", line 184, in <module>
    main()
  File "c:\Users\JoeriBloemen\OneDrive - Arxus.cloud\Arxus\LocalCoding\ASA\Billing_netw\asa_billing.py", line 70, in main
    print(device.get_s2s_rules("Imens01"))
  File "c:\Users\JoeriBloemen\OneDrive - Arxus.cloud\Arxus\LocalCoding\ASA\Billing_netw\arx_asa_test.py", line 55, in get_s2s_rules
    parsedout = parse_output(
  File "C:\Users\JoeriBloemen\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.9_qbz5n2kfra8p0\LocalCache\local-packages\Python39\site-packages\ntc_templates\parse.py", line 57, in parse_output
    cli_table.ParseCmd(data, attrs)
  File "C:\Users\JoeriBloemen\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.9_qbz5n2kfra8p0\LocalCache\local-packages\Python39\site-packages\textfsm\clitable.py", line 282, in ParseCmd
    self.table = self._ParseCmdItem(self.raw, template_file=template_files[0])
  File "C:\Users\JoeriBloemen\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.9_qbz5n2kfra8p0\LocalCache\local-packages\Python39\site-packages\textfsm\clitable.py", line 315, in _ParseCmdItem
    for record in fsm.ParseText(cmd_input):
  File "C:\Users\JoeriBloemen\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.9_qbz5n2kfra8p0\LocalCache\local-packages\Python39\site-packages\textfsm\parser.py", line 897, in ParseText
    self._CheckLine(line)
  File "C:\Users\JoeriBloemen\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.9_qbz5n2kfra8p0\LocalCache\local-packages\Python39\site-packages\textfsm\parser.py", line 946, in _CheckLine
    if self._Operations(rule, line):
  File "C:\Users\JoeriBloemen\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.9_qbz5n2kfra8p0\LocalCache\local-packages\Python39\site-packages\textfsm\parser.py", line 1026, in _Operations
    raise TextFSMError('State Error raised. Rule Line: %s. Input Line: %s'
textfsm.parser.TextFSMError: State Error raised. Rule Line: 146. Input Line:   Encryption   : AES-GCM-256            Hashing      : none
PS C:\Users\JoeriBloemen\OneDrive - Arxus.cloud\Arxus\LocalCoding>
ArxBloemJo commented 1 year ago

Connection : DefaultL2LGroup

Strings are not supported in the template. It's show a string because this is a dynamic cryptomap and connection peer is not a static ip

jvanderaa commented 1 year ago

There are a couple of things. First there is missing the connection type section at the top. Was this command executed with something more than just show vpn-sessiondb detail l2l?

ArxBloemJo commented 1 year ago

Oops i did not paste the whole output of the command (issue edited). Yes i use only the show vpn-sessiondb detail l2l command. I tested it locally if i change the template value CONNECTION: Currently """Value Filldown CONNECTION (\d+.\d+.\d+.\d+)""" TO """Value Filldown CONNECTION (.+?)"""

So it can accept everything on that value then i works and no error is raised.

mjbear commented 1 year ago

@ArxBloemJo @jvanderaa

Could use \S+ so that CONNECTION capture group doesn't inadvertently match white space.

The example above doesn't include Session Type which is required so this output won't parse as is.

mjbear commented 4 months ago

@ArxBloemJo Would you please respond back so we can gather more information to fix the ASA template?