search

zentures / sequence

(Unmaintained) High performance sequential log analyzer and parser
http://sequencer.io
517 stars 72 forks source link

ipv6 address not tokenized properly #2

Closed cryptix closed 9 years ago

cryptix commented 9 years ago

Hi,

i tried to add a rule for this log message from sshd.

msg:
Feb 06 13:37:00 box sshd[4388]: Accepted publickey for cryptix from dead:beef:1234:5678:223:32ff:feb1:2e50 port 58251 ssh2: RSA de:ad:be:ef:74:a6:bb:45:45:52:71:de:b2:12:34:56

rule:
%msgtime% %apphost% %appname% [ %sessionid% ] : Accepted publickey for %dstuser% from %srcipv6% port %integer% ssh2: RSA %string%

but i get Error (sequence: no pattern matched for this message).

I can't match the address nor the fingerprint because they are tokanized too much.

Here is what sequence scan -m returns for the message:

#   0: { Field="%funknown%", Type="%time%", Value="Feb 06 16:00:44" }
#   1: { Field="%funknown%", Type="%literal%", Value="higgs" }
#   2: { Field="%funknown%", Type="%literal%", Value="sshd" }
#   3: { Field="%funknown%", Type="%literal%", Value="[" }
#   4: { Field="%funknown%", Type="%integer%", Value="4388" }
#   5: { Field="%funknown%", Type="%literal%", Value="]" }
#   6: { Field="%funknown%", Type="%literal%", Value=":" }
#   7: { Field="%funknown%", Type="%literal%", Value="Accepted" }
#   8: { Field="%funknown%", Type="%literal%", Value="publickey" }
#   9: { Field="%funknown%", Type="%literal%", Value="for" }
#  10: { Field="%funknown%", Type="%literal%", Value="cryptix" }
#  11: { Field="%funknown%", Type="%literal%", Value="from" }
#  12: { Field="%funknown%", Type="%literal%", Value="dead" }
#  13: { Field="%funknown%", Type="%literal%", Value=":" }
#  14: { Field="%funknown%", Type="%literal%", Value="beef" }
#  15: { Field="%funknown%", Type="%literal%", Value=":" }
#  16: { Field="%funknown%", Type="%integer%", Value="1234" }
#  17: { Field="%funknown%", Type="%literal%", Value=":" }
#  18: { Field="%funknown%", Type="%integer%", Value="5678" }
#  19: { Field="%funknown%", Type="%literal%", Value=":" }
#  20: { Field="%funknown%", Type="%integer%", Value="223" }
#  21: { Field="%funknown%", Type="%literal%", Value=":" }
#  22: { Field="%funknown%", Type="%literal%", Value="32ff" }
#  23: { Field="%funknown%", Type="%literal%", Value=":" }
#  24: { Field="%funknown%", Type="%literal%", Value="feb1" }
#  25: { Field="%funknown%", Type="%literal%", Value=":" }
#  26: { Field="%funknown%", Type="%literal%", Value="2e50" }
#  27: { Field="%funknown%", Type="%literal%", Value="port" }
#  28: { Field="%funknown%", Type="%integer%", Value="58251" }
#  29: { Field="%funknown%", Type="%literal%", Value="ssh2" }
#  30: { Field="%funknown%", Type="%literal%", Value=":" }
#  31: { Field="%funknown%", Type="%literal%", Value="RSA" }
#  32: { Field="%funknown%", Type="%mac%", Value="de:ad:be:ef:74:a6" }
#  33: { Field="%funknown%", Type="%literal%", Value=":" }
#  34: { Field="%funknown%", Type="%mac%", Value="bb:45:45:52:71:de" }
#  35: { Field="%funknown%", Type="%literal%", Value=":" }
#  36: { Field="%funknown%", Type="%literal%", Value="b2" }
#  37: { Field="%funknown%", Type="%literal%", Value=":" }
#  38: { Field="%funknown%", Type="%integer%", Value="12" }
#  39: { Field="%funknown%", Type="%literal%", Value=":" }
#  40: { Field="%funknown%", Type="%integer%", Value="34" }
#  41: { Field="%funknown%", Type="%literal%", Value=":" }
#  42: { Field="%funknown%", Type="%integer%", Value="56" }

I would like to see this:

#   0: { Field="%funknown%", Type="%time%", Value="Feb 06 16:00:44" }
#   1: { Field="%funknown%", Type="%literal%", Value="higgs" }
#   2: { Field="%funknown%", Type="%literal%", Value="sshd" }
#   3: { Field="%funknown%", Type="%literal%", Value="[" }
#   4: { Field="%funknown%", Type="%integer%", Value="4388" }
#   5: { Field="%funknown%", Type="%literal%", Value="]" }
#   6: { Field="%funknown%", Type="%literal%", Value=":" }
#   7: { Field="%funknown%", Type="%literal%", Value="Accepted" }
#   8: { Field="%funknown%", Type="%literal%", Value="publickey" }
#   9: { Field="%funknown%", Type="%literal%", Value="for" }
#  10: { Field="%funknown%", Type="%literal%", Value="cryptix" }
#  11: { Field="%funknown%", Type="%literal%", Value="from" }
#  12: { Field="%funknown%", Type="%ipv6%", Value="2a02:8108:2140:6b64:223:32ff:feb1:2e50" }
#  13: { Field="%funknown%", Type="%literal%", Value="port" }
#  14: { Field="%funknown%", Type="%integer%", Value="58251" }
#  15: { Field="%funknown%", Type="%literal%", Value="ssh2" }
#  16: { Field="%funknown%", Type="%literal%", Value=":" }
#  17: { Field="%funknown%", Type="%literal%", Value="RSA" }
#  18: { Field="%funknown%", Type="%fingerprint%", Value="d1:93:fd:09:74:a6:bb:45:45:52:71:de:b2:38:9b:54" }

kind regards,

zhenjl commented 9 years ago

Thanks @cryptix!

Unfortunately the scanner is not able to handle IPv6 addresses yet because of the various formats it can take. I haven't settled on a good way to parse it without hammering the scanner performance. I am trying a few things and hope to get something done soon.

zhenjl commented 9 years ago 
go run ./sequence.go scan -m "Feb 06 13:37:00 box sshd[4388]: Accepted publickey for cryptix from dead:beef:1234:5678:223:32ff:feb1:2e50 port 58251 ssh2: RSA de:ad:be:ef:74:a6:bb:45:45:52:71:de:b2:12:34:56"
#   0: { Field="%funknown%", Type="%time%", Value="Feb 06 13:37:00", K=false, V=false }
#   1: { Field="%funknown%", Type="%literal%", Value="box", K=false, V=false }
#   2: { Field="%funknown%", Type="%literal%", Value="sshd", K=false, V=false }
#   3: { Field="%funknown%", Type="%literal%", Value="[", K=false, V=false }
#   4: { Field="%funknown%", Type="%integer%", Value="4388", K=false, V=false }
#   5: { Field="%funknown%", Type="%literal%", Value="]", K=false, V=false }
#   6: { Field="%funknown%", Type="%literal%", Value=":", K=false, V=false }
#   7: { Field="%funknown%", Type="%literal%", Value="Accepted", K=false, V=false }
#   8: { Field="%funknown%", Type="%literal%", Value="publickey", K=false, V=false }
#   9: { Field="%funknown%", Type="%literal%", Value="for", K=false, V=false }
#  10: { Field="%funknown%", Type="%literal%", Value="cryptix", K=false, V=false }
#  11: { Field="%funknown%", Type="%literal%", Value="from", K=false, V=false }
#  12: { Field="%funknown%", Type="%ipv6%", Value="dead:beef:1234:5678:223:32ff:feb1:2e50", K=false, V=false }
#  13: { Field="%funknown%", Type="%literal%", Value="port", K=false, V=false }
#  14: { Field="%funknown%", Type="%integer%", Value="58251", K=false, V=false }
#  15: { Field="%funknown%", Type="%literal%", Value="ssh2", K=false, V=false }
#  16: { Field="%funknown%", Type="%literal%", Value=":", K=false, V=false }
#  17: { Field="%funknown%", Type="%literal%", Value="RSA", K=false, V=false }
#  18: { Field="%funknown%", Type="%literal%", Value="de:ad:be:ef:74:a6:bb:45:45:52:71:de:b2:12:34:56", K=false, V=false }