Add TextFSM-Based Parsers to Mindwm

To improve the console output parsing functionality in Mindwm, we should add TextFSM-based patterns that can be used to parse console output. These patterns can be deployed as a Kafka consumer and producer, which will read events from tmux-pane-io-context and add metadata if the input and output match the specific patterns.

  io-context:
    - input: "^ip a show$"
      # https://github.com/networktocode/ntc-templates/blob/master/ntc_templates/templates/linux_ip_address_show.textfsm
      textfsm: |
         Value Required ID (\d+)
         Value Required INTERFACE ([^:]+)
         Value Required FLAGS (\S+)
         Value Required MTU (\d+)
         Value Required QDISC (\S+)
         Value Required STATE (\S+)
         Value Required GROUP (\S+)
         Value QLEN (\d+)
         Value MASTER (\S+)
         Value Required TYPE (\S+)
         Value ADDRESS ((?:[a-fA-F0-9:]{17}))
         Value BROADCAST ((?:[a-fA-F0-9:]{17}))
         Value List IP_ADDRESSES ([0-9\.]+)
         Value List IP_MASKS (\d{1,2})
         Value List IPV6_ADDRESSES ([0-9A-Fa-f:]+)
         Value List IPV6_MASKS (\d{1,3})

         Start
           ^\d+: -> Continue.Record
           ^\${ID}:\s+\${INTERFACE}:\s+<\${FLAGS}>\s+mtu\s+\${MTU}\s+qdisc\s+\${QDISC}(?:\s+master\s+\${MASTER})?\s+state\s+\${STATE}\s+group\s+\${GROUP}(?:\s+qlen\s+\${QLEN})?\s*$$
           ^\s+link/\${TYPE}(?:\s+\${ADDRESS}\s+brd\s+\${BROADCAST})?.*$$
           ^\s+altname.*$$
           ^\s+inet\s+\${IP_ADDRESSES}/\${IP_MASKS}\s+(:?brd|scope).*$$
           ^\s+inet6\s+\${IPV6_ADDRESSES}/\${IPV6_MASKS}\s+(?:brd|scope).*$$;;
           ^\s+valid_lft.*$$
           ^\s*$$
           ^. -> Error

For example, the following definition can be used to cover the "ip a show" output:

The following event in tmux-pane-io-context topic

{
  "host": "workstation",
  "input": "ip a show"
  "output": "1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: enp0s31f6: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc fq_codel state DOWN group default qlen 1000
    link/ether d4:93:90:19:f3:c7 brd ff:ff:ff:ff:ff:ff
3: wlp0s20f3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether c4:03:a8:c7:f5:33 brd ff:ff:ff:ff:ff:ff
    inet 10.232.200.252/22 brd 10.232.203.255 scope global dynamic noprefixroute wlp0s20f3
       valid_lft 84044sec preferred_lft 84044sec
    inet6 fe80::c61a:1683:821e:401d/64 scope link noprefixroute 
       valid_lft forever preferred_lft forever
4: br-02a5c284fcd0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN group default 
    link/ether 02:42:c2:e7:2c:0d brd ff:ff:ff:ff:ff:ff
    inet 172.19.0.1/16 brd 172.19.255.255 scope global br-02a5c284fcd0
       valid_lft forever preferred_lft forever
5: docker0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN group default 
    link/ether 02:42:dc:ba:a9:0a brd ff:ff:ff:ff:ff:ff
    inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0
       valid_lft forever preferred_lft forever
    inet6 fe80::42:dcff:feba:a90a/64 scope link 
       valid_lft forever preferred_lft forever
6: br-c4b5f3736bf8: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN group default 
    link/ether 02:42:3c:16:73:f0 brd ff:ff:ff:ff:ff:ff
    inet 192.168.49.1/24 brd 192.168.49.255 scope global br-c4b5f3736bf8
       valid_lft forever preferred_lft forever
"
  "metadata": {
    "shell": {
      "pid": "1123"
    },
    "tmux": {
      "pane_id": "23",
      "session_name": "example"
    },
    "window": {
      "id": "0x1abcdfegh",
      "name": "xterm"
    }
  },
}

After textFSM parser will be transformted to the following json document and produced to kafka topic

{
  "host": "workstation",
  "input": "ip a show"
  "output": "1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: enp0s31f6: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc fq_codel state DOWN group default qlen 1000
    link/ether d4:93:90:19:f3:c7 brd ff:ff:ff:ff:ff:ff
3: wlp0s20f3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether c4:03:a8:c7:f5:33 brd ff:ff:ff:ff:ff:ff
    inet 10.232.200.252/22 brd 10.232.203.255 scope global dynamic noprefixroute wlp0s20f3
       valid_lft 84044sec preferred_lft 84044sec
    inet6 fe80::c61a:1683:821e:401d/64 scope link noprefixroute 
       valid_lft forever preferred_lft forever
4: br-02a5c284fcd0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN group default 
    link/ether 02:42:c2:e7:2c:0d brd ff:ff:ff:ff:ff:ff
    inet 172.19.0.1/16 brd 172.19.255.255 scope global br-02a5c284fcd0
       valid_lft forever preferred_lft forever
5: docker0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN group default 
    link/ether 02:42:dc:ba:a9:0a brd ff:ff:ff:ff:ff:ff
    inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0
       valid_lft forever preferred_lft forever
    inet6 fe80::42:dcff:feba:a90a/64 scope link 
       valid_lft forever preferred_lft forever
6: br-c4b5f3736bf8: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN group default 
    link/ether 02:42:3c:16:73:f0 brd ff:ff:ff:ff:ff:ff
    inet 192.168.49.1/24 brd 192.168.49.255 scope global br-c4b5f3736bf8
       valid_lft forever preferred_lft forever
"
  "metadata": {
    "shell": {
      "pid": "1123"
    },
    "tmux": {
      "pane_id": "23",
      "session_name": "example"
    },
    "window": {
      "id": "0x1abcdfegh",
      "name": "xterm"
    },
    "objects": {
        "lo": { ...  },
        "enp0s31f6": { ... },
        "wlp0s20f3": { .... },         
        "br-02a5c284fcd0": { ... },
        "docker0": { .... }, 
        "br-c4b5f3736bf8:": { .... }
    } 

  },
}

metacoma / mindwm

Add TextFSM-Based Parsers to Mindwm #37