sh version of scripts/ipt-enable-logs.sh

pedropombeiro commented 2 years ago

What do you think about this version of scripts/ipt-enable-logs.sh for https://github.com/opustecnica/public/wiki/UDM-&-UDM-PRO-NOTES?

scripts/ipt-enable-logs.sh

#!/bin/sh

# [NOTE] Need to find a hook to run this script not only at startup but every time time the firewall is modified.

# Clear existing ipt-save if it exists.
FILE=/data/ipt-save
if test -f "${FILE}"; then
  rm -f "${FILE}"
fi

# Collect existing iptables configuration into an array.
prev_line=''
iptables-save |
while IFS= read -r line; do
  if [ -n "${prev_line}" ]; then
    ACTION="$(echo "${line}" | sed -E "s/.*-j\s(.*)$/\1/")"
    if echo "${ACTION}" | grep ^RETURN$ >/dev/null; then ACTION='ACCEPT'; fi
    echo -e "${prev_line}" | sed -E "s/^-A\sUBIOS_(\S+)\s.*-j LOG$/& --log-prefix \"[${ACTION}_\1] \"/" >> "${FILE}"
    prev_line=''
  fi

  if echo "${line}" | grep LOG$ >/dev/null
    then
      prev_line="${line}"
    else
      echo -e "${line}" >> "${FILE}"
      prev_line=''
  fi
done

on_boot.d/30-ipt-enable-logs-launch.sh

#!/bin/sh

set -e

/mnt/data/scripts/ipt-enable-logs.sh
iptables-restore -c < /data/ipt-save

I've compared the output of both the previous and new versions and they are identical.

pedropombeiro commented 2 years ago

By the way, it would be nice to see if you can get this added to https://github.com/boostchicken/udm-utilities, as this was the last remaining piece that was missing for me, and I only found out about this repo by chance.

pedropombeiro commented 2 years ago

Another (better) option would be be to use Go and compile the small program on the UDM using Docker:

/mnt/data/on_boot.d/30-ipt-enable-logs-launch.sh

#!/bin/sh

set -e

if ! iptables-save | grep '\--log-prefix "\[' > /dev/null; then
  /mnt/data/scripts/ipt-enable-logs.sh | iptables-restore -c
fi

/mnt/data/scripts/ipt-enable-logs/go.mod

module pedropombeiro.com/ipt-enable-logs

go 1.17

/mnt/data/scripts/ipt-enable-logs/main.go

package main

import (
    "fmt"
    "os"
    "os/exec"
    "regexp"
    "strconv"
    "strings"
)

func main() {
    cmd := exec.Command("iptables-save")
    outputBytes, err := cmd.Output()
    if err != nil {
        _ = fmt.Errorf("Failed to run iptables-save: %v", err)
        os.Exit(1)
    }

    str := string(outputBytes)
    lines := strings.Split(str, "\n")
    re := regexp.MustCompile(`-A UBIOS_([A-Z_]+) .* --comment (\d+) -j ([A-Z]+)`)

    for i, line := range lines {
        if !strings.HasSuffix(line, "-j LOG") {
            fmt.Println(line)
            continue
        }

        matches := re.FindSubmatch([]byte(lines[i+1]))
        commentNr, err := strconv.Atoi(string(matches[2]))
        if err != nil {
            commentNr = 0
        }
        actionName := getActionName(string(matches[3]))
        ruleName := getRuleName(string(matches[1]), commentNr)
        fmt.Printf(`%s --log-prefix "[%s-%s] "`+"\n", line, actionName, ruleName)
    }
}

func getActionName(action string) string {
    action = strings.Replace(action, "RETURN", "A", 1)
    action = strings.Replace(action, "REJECT", "R", 1)
    action = strings.Replace(action, "DROP", "D", 1)
    action = strings.Replace(action, "MASQUERADE", "M", 1)

    return action
}

func getRuleName(rule string, commentNr int) string {
  ruleName := strings.Replace(rule, "PREROUTING", "PRER", 1)
  ruleName = strings.Replace(ruleName, "POSTROUTING", "POSTR", 1)
  ruleName = strings.Replace(ruleName, "HOOK", "HK", 1)
  ruleName = strings.Replace(ruleName, "USER", "U", 1)
  if commentNr != 0 {
    ruleName = fmt.Sprintf("%s-%d", ruleName, commentNr & 0xFFFF)
  }
  return ruleName
}

This makes it easier to massage the data, so I can add the comment number to the rule ID to make it easier to trace it back to a iptables rule.

Snippet of iptables-save:

-A UBIOS_WAN_IN_USER -d 192.168.16.10/32 -p udp -m udp --dport 51820 -m conntrack --ctstate NEW -j LOG --log-prefix "[FW-A-WAN_IN_U-3010] "
-A UBIOS_WAN_IN_USER -d 192.168.16.10/32 -p udp -m udp --dport 51820 -m conntrack --ctstate NEW -m comment --comment 00000000008589937602 -j RETURN
-A UBIOS_WAN_IN_USER -d 192.168.16.10/32 -p udp -m udp --dport 51821 -m conntrack --ctstate NEW -j LOG --log-prefix "[FW-A-WAN_IN_U-3011] "
-A UBIOS_WAN_IN_USER -d 192.168.16.10/32 -p udp -m udp --dport 51821 -m conntrack --ctstate NEW -m comment --comment 00000000008589937603 -j RETURN

Snippet of logs:

[514604.992654] [FW-A-LAN_LOCAL_U-2000]IN=br0 OUT= MAC=26:5a:4c:a2:b1:02:78:4f:43:9c:74:3c:08:00 SRC=192.168.16.27 DST=192.168.16.1 LEN=52 TOS=0x08 PREC=0x40 TTL=64 ID=0 DF PROTO=TCP SPT=55706 DPT=22 WINDOW=397 RES=0x00 ACK URGP=0

Later on, I'm using two shell functions to make a presentable streaming log:

function loggatewayjson() {
  ssh gateway "tail -f /var/log/messages" | \
    rg "kernel:" | \
    sed "s/]IN/] IN/" | \
    jq --unbuffered -R '. | rtrimstr(" ") | split(": ") | {date: (.[0] | split(" ") | .[0:3] | join(" "))} + (.[1] | capture("\\[.+\\] \\[(?<rule>.*)\\].*")) + ((.[1] | capture("\\[.+\\] (?<rest>.*)") | .rest | split(" ") | map(select(startswith("[") == false) | split("=") | {(.[0]): .[1]})) | (reduce .[] as $item ({}; . + $item)))'
}

function loggateway() {
  loggatewayjson | jq --unbuffered -r '"\(.date) - \(.rule)\tIN=\(.IN)  \t\(.PROTO)\tSRC=\(.SRC)@\(.SPT)\tDST=\(.DST)@\(.DPT)\tLEN=\(.LEN)\t"'
}

loggateway output

Nov 14 10:58:31 - A-LAN_LOCAL_U-1097364144127   IN=br0      TCP SRC=192.168.16.10@55804 DST=192.168.16.1@443    LEN=52
Nov 14 10:58:31 - A-LAN_LOCAL_U-1097364144127   IN=br0      TCP SRC=192.168.16.10@55804 DST=192.168.16.1@443    LEN=52
Nov 14 10:58:31 - A-LAN_LOCAL_U-1097364144127   IN=br0      TCP SRC=192.168.16.10@55804 DST=192.168.16.1@443    LEN=52
Nov 14 10:58:31 - A-LAN_LOCAL_U-1097364144127   IN=br0      TCP SRC=192.168.16.10@55804 DST=192.168.16.1@443    LEN=52
Nov 14 10:58:31 - A-LAN_LOCAL_U-1097364144127   IN=br0      TCP SRC=192.168.16.10@55804 DST=192.168.16.1@443    LEN=52
Nov 14 10:58:31 - A-LAN_LOCAL_U-1097364144127   IN=br0      TCP SRC=192.168.16.10@55804 DST=192.168.16.1@443    LEN=52

boostchicken commented 2 years ago

@pedropombeiro please send a pull request to my repo for this. This is buried in the internet and nobody can find it haha

pedropombeiro commented 2 years ago

@pedropombeiro please send a pull request to my repo for this. This is buried in the internet and nobody can find it haha

Done: https://github.com/boostchicken/udm-utilities/pull/267

opustecnica / public