Logs enhancements - Githubissues

PaulSec commented 2 years ago

Hi everyone,

That project seems interesting. However, can logs be enhanced to get a proper format to ingest it into whatever {ELK, Splunk, whatever} platform?

At least, mandatory things would be:

scanning time (timestamp)
source ip address
interface which got the packets to
probe

Just a quick search and I found this which might be interesting : https://rust-lang-nursery.github.io/rust-cookbook/development_tools/debugging/config_log.html

Thanks for that project, looking forward to it.

Frky commented 2 years ago

Thank you for the feedback. Improving logs and making them compatible with other tools is indeed something we should do. Thank you for the suggestion, it's added to the TODO.

p-l- commented 2 years ago

Here is an example of Zeek / passiverecon log, generated by Masscanned tests (tab separated):

#separator \x09
#set_separator  ,
#empty_field    (empty)
#unset_field    -
#path   /dev/stdout
#open   2021-12-09-15-48-33
#fields ts      uid     host    srvport recon_type      source  value   targetval
#types  time    string  addr    port    enum    string  string  string
1639061313.373192       -       192.0.0.0       -       PassiveRecon::MAC_ADDRESS       ARP_REQUEST_SRC 0a:e0:a1:e2:97:7f       -
1639061313.373466       -       192.0.0.0       -       PassiveRecon::MAC_ADDRESS       ARP_REPLY_DST   0a:e0:a1:e2:97:7f       -
1639061313.375568       -       192.0.0.0       -       PassiveRecon::MAC_ADDRESS       ARP_REQUEST_SRC 0a:e0:a1:e2:97:7f       -
1639061317.468172       C6mDtd4ol4TQXGkkD4      192.0.0.0       -       PassiveRecon::TCP_HONEYPOT_HIT  tcp/1152        (empty) -
1639061317.471396       CfGWYr1Wb6TpPvEqKe      192.0.0.0       -       PassiveRecon::TCP_HONEYPOT_HIT  tcp/2003        (empty) -
1639061317.474237       CnGKjs1DFOzrNKt047      192.0.0.0       -       PassiveRecon::TCP_HONEYPOT_HIT  tcp/2193        (empty) -
1639061317.476060       CFecpS1QB8jEK5TkC1      192.0.0.0       -       PassiveRecon::TCP_HONEYPOT_HIT  tcp/3709        (empty) -
1639061317.477831       CpdYYn2ZXOeipF3EQl      192.0.0.0       -       PassiveRecon::TCP_HONEYPOT_HIT  tcp/4054        (empty) -
1639061317.479621       CQkPXf400nWgNhY7Oj      192.0.0.0       -       PassiveRecon::TCP_HONEYPOT_HIT  tcp/6605        (empty) -
1639061317.481791       CeJSvo4odEzzJABwef      192.0.0.0       -       PassiveRecon::TCP_HONEYPOT_HIT  tcp/6737        (empty) -
1639061317.484001       CkL2zA4yXskNSHjMEb      192.0.0.0       -       PassiveRecon::TCP_HONEYPOT_HIT  tcp/6875        (empty) -
1639061317.485789       CSOrgV3oKknguUEEUk      192.0.0.0       -       PassiveRecon::TCP_HONEYPOT_HIT  tcp/7320        (empty) -
1639061317.487549       CWQINQ1Zlf6E6rQU2g      192.0.0.0       -       PassiveRecon::TCP_HONEYPOT_HIT  tcp/8898        (empty) -
1639061317.489292       CdaWkq2azN9cXoGxmk      192.0.0.0       -       PassiveRecon::TCP_HONEYPOT_HIT  tcp/9513        (empty) -
1639061317.491086       CakOTR12Er81Lgoga2      192.0.0.0       -       PassiveRecon::TCP_HONEYPOT_HIT  tcp/9738        (empty) -
1639061317.492896       C5pxI24esH6TZbuzt3      192.0.0.0       -       PassiveRecon::TCP_HONEYPOT_HIT  tcp/10623       (empty) -
1639061317.494865       C9AuoZ2SYZYEregp91      192.0.0.0       -       PassiveRecon::TCP_HONEYPOT_HIT  tcp/10723       (empty) -
1639061317.496588       CznmuI2FKbvhcmTfRb      192.0.0.0       -       PassiveRecon::TCP_HONEYPOT_HIT  tcp/11253       (empty) -
1639061317.498370       CbEAdI2qRf9Cff0lYe      192.0.0.0       -       PassiveRecon::TCP_HONEYPOT_HIT  tcp/12125       (empty) -
1639061317.500136       CgE4ZI24wte7h0kUIh      192.0.0.0       -       PassiveRecon::TCP_HONEYPOT_HIT  tcp/12189       (empty) -
1639061317.501907       CqNySyCNHYifWuEb1       192.0.0.0       -       PassiveRecon::TCP_HONEYPOT_HIT  tcp/12873       (empty) -
1639061317.503719       CrnjLX2ytpcu6GBQB3      192.0.0.0       -       PassiveRecon::TCP_HONEYPOT_HIT  tcp/14648       (empty) -
1639061317.505540       C37dyO1Nq5CaiuhJs6      192.0.0.0       -       PassiveRecon::TCP_HONEYPOT_HIT  tcp/14659       (empty) -
1639061317.507534       Covj4Y2RVLV8UNRfqg      192.0.0.0       -       PassiveRecon::TCP_HONEYPOT_HIT  tcp/16242       (empty) -
1639061317.509371       CJtDcG2EzgzY5SCPvh      192.0.0.0       -       PassiveRecon::TCP_HONEYPOT_HIT  tcp/16243       (empty) -
1639061317.511202       CVqdVB4y32PGUN1rze      192.0.0.0       -       PassiveRecon::TCP_HONEYPOT_HIT  tcp/17209       (empty) -
1639061317.513007       Cj5xzfDPDCTButGEc       192.0.0.0       -       PassiveRecon::TCP_HONEYPOT_HIT  tcp/17492       (empty) -
1639061317.514815       CjMv8k8CuDaY8bb6a       192.0.0.0       -       PassiveRecon::TCP_HONEYPOT_HIT  tcp/17667       (empty) -
1639061317.516584       Co6Ogk1OVC64iOCjc3      192.0.0.0       -       PassiveRecon::TCP_HONEYPOT_HIT  tcp/17838       (empty) -
1639061317.518665       CbKbtK3aqLzjPGepKg      192.0.0.0       -       PassiveRecon::TCP_HONEYPOT_HIT  tcp/18081       (empty) -
1639061317.520494       C8wSIP3onIjpPXxxok      192.0.0.0       -       PassiveRecon::TCP_HONEYPOT_HIT  tcp/18682       (empty) -
1639061317.522240       CBU8Mz2JTNWOzn3el5      192.0.0.0       -       PassiveRecon::TCP_HONEYPOT_HIT  tcp/18790       (empty) -
1639061317.524030       CiLwg62BuSaAmuj7t8      192.0.0.0       -       PassiveRecon::TCP_HONEYPOT_HIT  tcp/19124       (empty) -
1639061317.525890       CszEI24Ku7ZUaxrpE8      192.0.0.0       -       PassiveRecon::TCP_HONEYPOT_HIT  tcp/19288       (empty) -
1639061317.527598       C7RIDs2WTw4HPPzkAh      192.0.0.0       -       PassiveRecon::TCP_HONEYPOT_HIT  tcp/19558       (empty) -
1639061317.529321       CSQSwL2svHFR7F8oAd      192.0.0.0       -       PassiveRecon::TCP_HONEYPOT_HIT  tcp/19628       (empty) -
1639061317.531277       CNPOLc2mJbnyIvxst       192.0.0.0       -       PassiveRecon::TCP_HONEYPOT_HIT  tcp/19789       (empty) -
1639061317.533008       ChLZxQ2LkVlJ2qjac5      192.0.0.0       -       PassiveRecon::TCP_HONEYPOT_HIT  tcp/20093       (empty) -
1639061317.534788       CB6Az5Cam1M6hpSZ6       192.0.0.0       -       PassiveRecon::TCP_HONEYPOT_HIT  tcp/21014       (empty) -
1639061317.536546       ChtQBp3ZuXXfENXlt7      192.0.0.0       -       PassiveRecon::TCP_HONEYPOT_HIT  tcp/21459       (empty) -
1639061317.538351       CaD7h1c7NqhWnssEb       192.0.0.0       -       PassiveRecon::TCP_HONEYPOT_HIT  tcp/21740       (empty) -
1639061317.540148       CZnIZv1DN9v1SRRpG4      192.0.0.0       -       PassiveRecon::TCP_HONEYPOT_HIT  tcp/24070       (empty) -
1639061317.543128       C0MrVJ29oUhG3nouP       192.0.0.0       -       PassiveRecon::TCP_HONEYPOT_HIT  tcp/24312       (empty) -
1639061317.544966       Ce49XZ2l8ox33nWFe1      192.0.0.0       -       PassiveRecon::TCP_HONEYPOT_HIT  tcp/24576       (empty) -
1639061317.546720       Ce3rJY2TP8xx2jKkKd      192.0.0.0       -       PassiveRecon::TCP_HONEYPOT_HIT  tcp/26939       (empty) -
1639061317.548423       CrKQWc29fFt1KQDGA       192.0.0.0       -       PassiveRecon::TCP_HONEYPOT_HIT  tcp/27136       (empty) -
1639061317.550122       C16Ui614jNXuZEwyn4      192.0.0.0       -       PassiveRecon::TCP_HONEYPOT_HIT  tcp/27165       (empty) -
1639061317.551896       CasvJ03mUTlWlCagF7      192.0.0.0       -       PassiveRecon::TCP_HONEYPOT_HIT  tcp/27361       (empty) -
1639061317.553605       CwplXy1LTRfQ7mgKOk      192.0.0.0       -       PassiveRecon::TCP_HONEYPOT_HIT  tcp/29971       (empty) -
1639061317.555539       CMis6S1jzpr9uCNwij      192.0.0.0       -       PassiveRecon::TCP_HONEYPOT_HIT  tcp/31088       (empty) -
1639061317.557395       CtOzNv3yGIJnRMpRMc      192.0.0.0       -       PassiveRecon::TCP_HONEYPOT_HIT  tcp/33011       (empty) -
1639061317.559203       CmSBvuByUvmfD9tPd       192.0.0.0       -       PassiveRecon::TCP_HONEYPOT_HIT  tcp/33068       (empty) -
1639061317.560904       CSoKow3SLNMIOgdMm3      192.0.0.0       -       PassiveRecon::TCP_HONEYPOT_HIT  tcp/34990       (empty) -
1639061317.562629       CZHZyi4Ygo68bYIXbd      192.0.0.0       -       PassiveRecon::TCP_HONEYPOT_HIT  tcp/35093       (empty) -
1639061317.564380       CHBBLl40lAe6kOPb3a      192.0.0.0       -       PassiveRecon::TCP_HONEYPOT_HIT  tcp/35958       (empty) -
1639061317.566288       CxoVN82Q8GlQ2w5JGc      192.0.0.0       -       PassiveRecon::TCP_HONEYPOT_HIT  tcp/36626       (empty) -
1639061317.568060       CJZvwC3XmOSuHtzMf       192.0.0.0       -       PassiveRecon::TCP_HONEYPOT_HIT  tcp/36789       (empty) -
1639061317.569753       CTJ0uZDz9OW8zK0wh       192.0.0.0       -       PassiveRecon::TCP_HONEYPOT_HIT  tcp/37130       (empty) -
1639061317.571541       CqFEud3ocu5OtI24p7      192.0.0.0       -       PassiveRecon::TCP_HONEYPOT_HIT  tcp/37238       (empty) -
1639061317.573305       CYtnq129UIrXr1crzd      192.0.0.0       -       PassiveRecon::TCP_HONEYPOT_HIT  tcp/37256       (empty) -
1639061317.575082       CdS7qO3go48RUnsEba      192.0.0.0       -       PassiveRecon::TCP_HONEYPOT_HIT  tcp/37697       (empty) -
1639061317.576928       Cly0W73NYsWORGnDS8      192.0.0.0       -       PassiveRecon::TCP_HONEYPOT_HIT  tcp/37890       (empty) -
1639061317.579049       CyGrsW1JlY37zflS5j      192.0.0.0       -       PassiveRecon::TCP_HONEYPOT_HIT  tcp/38958       (empty) -
1639061317.580882       CoKd9m1qHC6gqLXt2h      192.0.0.0       -       PassiveRecon::TCP_HONEYPOT_HIT  tcp/42131       (empty) -
1639061317.582747       CRTHl22SOHhT3Ghvjk      192.0.0.0       -       PassiveRecon::TCP_HONEYPOT_HIT  tcp/43864       (empty) -
1639061317.584653       CadMgK1EgDKfWJKMSg      192.0.0.0       -       PassiveRecon::TCP_HONEYPOT_HIT  tcp/44420       (empty) -
1639061317.586544       CiL07kmA477YGJXrj       192.0.0.0       -       PassiveRecon::TCP_HONEYPOT_HIT  tcp/44655       (empty) -
1639061317.588529       CmLCOh4fSXkKxlgYNl      192.0.0.0       -       PassiveRecon::TCP_HONEYPOT_HIT  tcp/44868       (empty) -
1639061317.590675       ClnPPX2AufOeubUFb7      192.0.0.0       -       PassiveRecon::TCP_HONEYPOT_HIT  tcp/45157       (empty) -
1639061317.592575       CK5uir4qCHa1onSG9b      192.0.0.0       -       PassiveRecon::TCP_HONEYPOT_HIT  tcp/46213       (empty) -
1639061317.594371       CMP21h2LgGFzitN4kk      192.0.0.0       -       PassiveRecon::TCP_HONEYPOT_HIT  tcp/46497       (empty) -
1639061317.596167       CNRD7mGXdseX0Gljb       192.0.0.0       -       PassiveRecon::TCP_HONEYPOT_HIT  tcp/46955       (empty) -
1639061317.598025       C8JByk1pKR2zzyktz       192.0.0.0       -       PassiveRecon::TCP_HONEYPOT_HIT  tcp/49049       (empty) -
1639061317.599844       CCVYqZ0Dn6S1nlwQh       192.0.0.0       -       PassiveRecon::TCP_HONEYPOT_HIT  tcp/49067       (empty) -
1639061317.601665       Csfr8n46Pzhqk2mrZh      192.0.0.0       -       PassiveRecon::TCP_HONEYPOT_HIT  tcp/49452       (empty) -
1639061317.603682       Cgo6rp1mWKx25OEyq3      192.0.0.0       -       PassiveRecon::TCP_HONEYPOT_HIT  tcp/49480       (empty) -
1639061317.605605       CRBOVX3P1lSZSXqCN8      192.0.0.0       -       PassiveRecon::TCP_HONEYPOT_HIT  tcp/50498       (empty) -
1639061317.607380       C2I0yF1DDgRnn8veQa      192.0.0.0       -       PassiveRecon::TCP_HONEYPOT_HIT  tcp/50945       (empty) -
1639061317.609312       CMQJRj1DPYWHnp6yba      192.0.0.0       -       PassiveRecon::TCP_HONEYPOT_HIT  tcp/51181       (empty) -
1639061317.611233       Ckl4l9T3Qw7yxCdMk       192.0.0.0       -       PassiveRecon::TCP_HONEYPOT_HIT  tcp/52890       (empty) -
1639061317.613095       C0qDlL1WYVfEXKm 192.0.0.0       -       PassiveRecon::TCP_HONEYPOT_HIT  tcp/53301       (empty) -
1639061317.615100       CzZUN720i1W0FsrhZg      192.0.0.0       -       PassiveRecon::TCP_HONEYPOT_HIT  tcp/53407       (empty) -
1639061317.616973       C3WamZ3n3k209el9Ld      192.0.0.0       -       PassiveRecon::TCP_HONEYPOT_HIT  tcp/53417       (empty) -
1639061317.618835       C2qfxu4cTLnXY4uaL8      192.0.0.0       -       PassiveRecon::TCP_HONEYPOT_HIT  tcp/53980       (empty) -
1639061317.620630       C3WrX53osbwLYx8jZ4      192.0.0.0       -       PassiveRecon::TCP_HONEYPOT_HIT  tcp/55827       (empty) -
1639061317.622447       CBsjqq3imoEUSeoj93      192.0.0.0       -       PassiveRecon::TCP_HONEYPOT_HIT  tcp/56483       (empty) -
1639061317.624360       CWRgYx4705rJ0ui0l1      192.0.0.0       -       PassiveRecon::TCP_HONEYPOT_HIT  tcp/58552       (empty) -
1639061317.626212       CdguYXnLpFX4OL9Jg       192.0.0.0       -       PassiveRecon::TCP_HONEYPOT_HIT  tcp/58713       (empty) -
1639061317.628229       Cqa6E7RQwnmC3yHQg       192.0.0.0       -       PassiveRecon::TCP_HONEYPOT_HIT  tcp/58836       (empty) -
1639061317.630042       CMiiyX3tNPv6a2TuJ9      192.0.0.0       -       PassiveRecon::TCP_HONEYPOT_HIT  tcp/59362       (empty) -
1639061317.631935       CFHCWX1YbwVJB8mZ62      192.0.0.0       -       PassiveRecon::TCP_HONEYPOT_HIT  tcp/59560       (empty) -
1639061317.633753       C8H3Rf1cNXKokY2pe2      192.0.0.0       -       PassiveRecon::TCP_HONEYPOT_HIT  tcp/60534       (empty) -
1639061317.635538       CtmrbqchMSjcOxtQg       192.0.0.0       -       PassiveRecon::TCP_HONEYPOT_HIT  tcp/60555       (empty) -
1639061317.637352       CVzkLx3L12WEe7V8A1      192.0.0.0       -       PassiveRecon::TCP_HONEYPOT_HIT  tcp/60660       (empty) -
1639061317.639398       CyipUb3RsmyCFQv5Hg      192.0.0.0       -       PassiveRecon::TCP_HONEYPOT_HIT  tcp/61615       (empty) -
1639061317.641233       CjIsW13yP614yYO3M9      192.0.0.0       -       PassiveRecon::TCP_HONEYPOT_HIT  tcp/62402       (empty) -
1639061317.643038       C4NyqWTTDoivyq0sb       192.0.0.0       -       PassiveRecon::TCP_HONEYPOT_HIT  tcp/62533       (empty) -
1639061317.644900       CAEidk2OVOGgIuF7d7      192.0.0.0       -       PassiveRecon::TCP_HONEYPOT_HIT  tcp/62941       (empty) -
1639061317.646718       C94cUJ8h3qeowLcol       192.0.0.0       -       PassiveRecon::TCP_HONEYPOT_HIT  tcp/63240       (empty) -
1639061317.648467       CLYZIx3W9GKa5rcB24      192.0.0.0       -       PassiveRecon::TCP_HONEYPOT_HIT  tcp/63339       (empty) -
1639061317.650295       CV8r7a3jKFVFbDaf69      192.0.0.0       -       PassiveRecon::TCP_HONEYPOT_HIT  tcp/63616       (empty) -
1639061317.652363       CxBmKv43ER76crKw87      192.0.0.0       -       PassiveRecon::TCP_HONEYPOT_HIT  tcp/64380       (empty) -
1639061317.654168       CyC2Df1xkWemM2qJt7      192.0.0.0       -       PassiveRecon::TCP_HONEYPOT_HIT  tcp/65438       (empty) -
1639061318.660542       CQ5gME2Cz2Obj0iytg      192.0.0.0       -       PassiveRecon::TCP_HONEYPOT_HIT  tcp/445 (empty) -
1639061320.798501       CKxnqi4UukEwGDnEKl      2001:41d0::1234:5678    -       PassiveRecon::TCP_HONEYPOT_HIT  tcp/445 (empty) -
1639061321.823681       Cib9Td27u9TipYu0cg      192.0.0.0       -       PassiveRecon::TCP_HONEYPOT_HIT  tcp/80  (empty) -
1639061321.823681       Cib9Td27u9TipYu0cg      192.0.0.0       -       PassiveRecon::TCP_HONEYPOT_HIT  tcp/80  GET / HTTP/1.1\x0d\x0a\x0d\x0a  -
1639061321.823681       Cib9Td27u9TipYu0cg      192.0.0.0       -       PassiveRecon::HTTP_HONEYPOT_REQUEST     GET-1.1-tcp/80  /       -
1639061322.869884       Cfzulb3ibaHn3j2Xuc      192.0.0.0       -       PassiveRecon::TCP_HONEYPOT_HIT  tcp/443 (empty) -
1639061322.869884       Cfzulb3ibaHn3j2Xuc      192.0.0.0       -       PassiveRecon::TCP_HONEYPOT_HIT  tcp/443 GET / HTTP/1.1\x0d\x0a\x0d\x0a  -
1639061322.869884       Cfzulb3ibaHn3j2Xuc      192.0.0.0       -       PassiveRecon::HTTP_HONEYPOT_REQUEST     GET-1.1-tcp/443 /       -
1639061323.884783       CLa3Vu4jYNxX3STVk       192.0.0.0       -       PassiveRecon::TCP_HONEYPOT_HIT  tcp/5000        (empty) -
1639061323.884783       CLa3Vu4jYNxX3STVk       192.0.0.0       -       PassiveRecon::TCP_HONEYPOT_HIT  tcp/5000        GET / HTTP/1.1\x0d\x0a\x0d\x0a  -
1639061323.884783       CLa3Vu4jYNxX3STVk       192.0.0.0       -       PassiveRecon::HTTP_HONEYPOT_REQUEST     GET-1.1-tcp/5000        /       -
1639061324.912634       CrsaHHNO0mglyKR8f       192.0.0.0       -       PassiveRecon::TCP_HONEYPOT_HIT  tcp/53228       (empty) -
1639061324.912634       CrsaHHNO0mglyKR8f       192.0.0.0       -       PassiveRecon::TCP_HONEYPOT_HIT  tcp/53228       GET / HTTP/1.1\x0d\x0a\x0d\x0a  -
1639061324.912634       CrsaHHNO0mglyKR8f       192.0.0.0       -       PassiveRecon::HTTP_HONEYPOT_REQUEST     GET-1.1-tcp/53228       /       -
1639061325.941867       CkOvDv1TryzFWbqVqg      2001:41d0::1234:5678    -       PassiveRecon::TCP_HONEYPOT_HIT  tcp/80  (empty) -
1639061325.941867       CkOvDv1TryzFWbqVqg      2001:41d0::1234:5678    -       PassiveRecon::TCP_HONEYPOT_HIT  tcp/80  GET / HTTP/1.1\x0d\x0a\x0d\x0a  -
1639061325.941867       CkOvDv1TryzFWbqVqg      2001:41d0::1234:5678    -       PassiveRecon::HTTP_HONEYPOT_REQUEST     GET-1.1-tcp/80  /       -
1639061326.966666       CC5Ceh1EoKgd8Ukp4c      2001:41d0::1234:5678    -       PassiveRecon::TCP_HONEYPOT_HIT  tcp/443 (empty) -
1639061326.966666       CC5Ceh1EoKgd8Ukp4c      2001:41d0::1234:5678    -       PassiveRecon::TCP_HONEYPOT_HIT  tcp/443 GET / HTTP/1.1\x0d\x0a\x0d\x0a  -
1639061326.966666       CC5Ceh1EoKgd8Ukp4c      2001:41d0::1234:5678    -       PassiveRecon::HTTP_HONEYPOT_REQUEST     GET-1.1-tcp/443 /       -
1639061327.998009       CyhJoh3CMsTvR5knpc      2001:41d0::1234:5678    -       PassiveRecon::TCP_HONEYPOT_HIT  tcp/5000        (empty) -
1639061327.998009       CyhJoh3CMsTvR5knpc      2001:41d0::1234:5678    -       PassiveRecon::TCP_HONEYPOT_HIT  tcp/5000        GET / HTTP/1.1\x0d\x0a\x0d\x0a  -
1639061327.998009       CyhJoh3CMsTvR5knpc      2001:41d0::1234:5678    -       PassiveRecon::HTTP_HONEYPOT_REQUEST     GET-1.1-tcp/5000        /       -
1639061329.022496       CRDMxDbHmgBOfOnHk       2001:41d0::1234:5678    -       PassiveRecon::TCP_HONEYPOT_HIT  tcp/53228       (empty) -
1639061329.022496       CRDMxDbHmgBOfOnHk       2001:41d0::1234:5678    -       PassiveRecon::TCP_HONEYPOT_HIT  tcp/53228       GET / HTTP/1.1\x0d\x0a\x0d\x0a  -
1639061329.022496       CRDMxDbHmgBOfOnHk       2001:41d0::1234:5678    -       PassiveRecon::HTTP_HONEYPOT_REQUEST     GET-1.1-tcp/53228       /       -
1639061330.080737       C30ioKOexmAColSmc       192.0.0.0       -       PassiveRecon::TCP_HONEYPOT_HIT  tcp/80  (empty) -
1639061330.080737       C30ioKOexmAColSmc       192.0.0.0       -       PassiveRecon::TCP_HONEYPOT_HIT  tcp/80  OPTIONS -
1639061331.095348       CcUOnR2HCoAYBcaqJl      192.0.0.0       -       PassiveRecon::TCP_HONEYPOT_HIT  tcp/443 (empty) -
1639061331.095348       CcUOnR2HCoAYBcaqJl      192.0.0.0       -       PassiveRecon::TCP_HONEYPOT_HIT  tcp/443 OPTIONS -
1639061332.114060       CHxHqX3WhO3eMc4vAg      192.0.0.0       -       PassiveRecon::TCP_HONEYPOT_HIT  tcp/5000        (empty) -
1639061332.114060       CHxHqX3WhO3eMc4vAg      192.0.0.0       -       PassiveRecon::TCP_HONEYPOT_HIT  tcp/5000        OPTIONS -
1639061333.131957       CEYiLU2Bn6vw9Gb3y4      192.0.0.0       -       PassiveRecon::TCP_HONEYPOT_HIT  tcp/53228       (empty) -
1639061333.131957       CEYiLU2Bn6vw9Gb3y4      192.0.0.0       -       PassiveRecon::TCP_HONEYPOT_HIT  tcp/53228       OPTIONS -
1639061338.175654       C9TwxW2Fg5Zj21uLf2      2001:41d0::1234:5678    -       PassiveRecon::TCP_HONEYPOT_HIT  tcp/80  (empty) -
1639061338.175654       C9TwxW2Fg5Zj21uLf2      2001:41d0::1234:5678    -       PassiveRecon::TCP_HONEYPOT_HIT  tcp/80  OPTIONS -
1639061339.193413       CJfluvd90DORXkDke       2001:41d0::1234:5678    -       PassiveRecon::TCP_HONEYPOT_HIT  tcp/443 (empty) -
1639061339.193413       CJfluvd90DORXkDke       2001:41d0::1234:5678    -       PassiveRecon::TCP_HONEYPOT_HIT  tcp/443 OPTIONS -
1639061340.209988       Cb4VXQ6QsPYpISfFa       2001:41d0::1234:5678    -       PassiveRecon::TCP_HONEYPOT_HIT  tcp/5000        (empty) -
1639061340.209988       Cb4VXQ6QsPYpISfFa       2001:41d0::1234:5678    -       PassiveRecon::TCP_HONEYPOT_HIT  tcp/5000        OPTIONS -
1639061341.240618       CdCkLR2xTMwJj0dluj      2001:41d0::1234:5678    -       PassiveRecon::TCP_HONEYPOT_HIT  tcp/53228       (empty) -
1639061341.240618       CdCkLR2xTMwJj0dluj      2001:41d0::1234:5678    -       PassiveRecon::TCP_HONEYPOT_HIT  tcp/53228       OPTIONS -
1639061346.579012       C6mThd1ycEv6YyptQc      192.0.0.0       -       PassiveRecon::TCP_HONEYPOT_HIT  tcp/22  (empty) -
1639061346.579012       C6mThd1ycEv6YyptQc      192.0.0.0       -       PassiveRecon::TCP_HONEYPOT_HIT  tcp/22  SSH-2.0-AsyncSSH_2.1.0\x0d\x0a  -
1639061346.579012       C6mThd1ycEv6YyptQc      192.0.0.0       -       PassiveRecon::SSH_CLIENT        -       SSH-2.0-AsyncSSH_2.1.0  -
1639061347.588763       CwcuJn3h6asufSGeIl      192.0.0.0       -       PassiveRecon::TCP_HONEYPOT_HIT  tcp/80  (empty) -
1639061347.588763       CwcuJn3h6asufSGeIl      192.0.0.0       -       PassiveRecon::TCP_HONEYPOT_HIT  tcp/80  SSH-2.0-PuTTY\x0d\x0a   -
1639061347.588763       CwcuJn3h6asufSGeIl      192.0.0.0       -       PassiveRecon::SSH_CLIENT        -       SSH-2.0-PuTTY   -
1639061348.602535       Cu6F4L2WQ7JkVh0bnf      192.0.0.0       -       PassiveRecon::TCP_HONEYPOT_HIT  tcp/2222        (empty) -
1639061348.602535       Cu6F4L2WQ7JkVh0bnf      192.0.0.0       -       PassiveRecon::TCP_HONEYPOT_HIT  tcp/2222        SSH-2.0-libssh2_1.4.3\x0d\x0a   -
1639061348.602535       Cu6F4L2WQ7JkVh0bnf      192.0.0.0       -       PassiveRecon::SSH_CLIENT        -       SSH-2.0-libssh2_1.4.3   -
1639061349.612657       CjisOh368EdesLr5e5      192.0.0.0       -       PassiveRecon::TCP_HONEYPOT_HIT  tcp/2022        (empty) -
1639061349.612657       CjisOh368EdesLr5e5      192.0.0.0       -       PassiveRecon::TCP_HONEYPOT_HIT  tcp/2022        SSH-2.0-Go\x0d\x0a      -
1639061349.612657       CjisOh368EdesLr5e5      192.0.0.0       -       PassiveRecon::SSH_CLIENT        -       SSH-2.0-Go      -
1639061350.628069       C7Foln34xvOpf8nMli      192.0.0.0       -       PassiveRecon::TCP_HONEYPOT_HIT  tcp/23874       (empty) -
1639061350.628069       C7Foln34xvOpf8nMli      192.0.0.0       -       PassiveRecon::TCP_HONEYPOT_HIT  tcp/23874       SSH-2.0-PUTTY\x0d\x0a   -
1639061350.628069       C7Foln34xvOpf8nMli      192.0.0.0       -       PassiveRecon::SSH_CLIENT        -       SSH-2.0-PUTTY   -
1639061351.642122       CtgQx43sIwhjXhMbqh      192.0.0.0       -       PassiveRecon::TCP_HONEYPOT_HIT  tcp/50000       (empty) -
1639061351.642122       CtgQx43sIwhjXhMbqh      192.0.0.0       -       PassiveRecon::TCP_HONEYPOT_HIT  tcp/50000       SSH-2.0-AsyncSSH_2.1.0\x0d\x0a  -
1639061351.642122       CtgQx43sIwhjXhMbqh      192.0.0.0       -       PassiveRecon::SSH_CLIENT        -       SSH-2.0-AsyncSSH_2.1.0  -
1639061352.695615       Ch6Xfn1iTm6uy0NaGg      2001:41d0::1234:5678    -       PassiveRecon::TCP_HONEYPOT_HIT  tcp/22  (empty) -
1639061352.695615       Ch6Xfn1iTm6uy0NaGg      2001:41d0::1234:5678    -       PassiveRecon::TCP_HONEYPOT_HIT  tcp/22  SSH-2.0-AsyncSSH_2.1.0\x0d\x0a  -
1639061352.695615       Ch6Xfn1iTm6uy0NaGg      2001:41d0::1234:5678    -       PassiveRecon::SSH_CLIENT        -       SSH-2.0-AsyncSSH_2.1.0  -
1639061353.712083       C8kPbk48zce9pcBEc       2001:41d0::1234:5678    -       PassiveRecon::TCP_HONEYPOT_HIT  tcp/80  (empty) -
1639061353.712083       C8kPbk48zce9pcBEc       2001:41d0::1234:5678    -       PassiveRecon::TCP_HONEYPOT_HIT  tcp/80  SSH-2.0-PuTTY\x0d\x0a   -
1639061353.712083       C8kPbk48zce9pcBEc       2001:41d0::1234:5678    -       PassiveRecon::SSH_CLIENT        -       SSH-2.0-PuTTY   -
1639061354.727970       Cy1nGc5VBPHUAx72l       2001:41d0::1234:5678    -       PassiveRecon::TCP_HONEYPOT_HIT  tcp/2222        (empty) -
1639061354.727970       Cy1nGc5VBPHUAx72l       2001:41d0::1234:5678    -       PassiveRecon::TCP_HONEYPOT_HIT  tcp/2222        SSH-2.0-libssh2_1.4.3\x0d\x0a   -
1639061354.727970       Cy1nGc5VBPHUAx72l       2001:41d0::1234:5678    -       PassiveRecon::SSH_CLIENT        -       SSH-2.0-libssh2_1.4.3   -
1639061355.744969       CWUHB14pvfFq1wQ1n6      2001:41d0::1234:5678    -       PassiveRecon::TCP_HONEYPOT_HIT  tcp/2022        (empty) -
1639061355.744969       CWUHB14pvfFq1wQ1n6      2001:41d0::1234:5678    -       PassiveRecon::TCP_HONEYPOT_HIT  tcp/2022        SSH-2.0-Go\x0d\x0a      -
1639061355.744969       CWUHB14pvfFq1wQ1n6      2001:41d0::1234:5678    -       PassiveRecon::SSH_CLIENT        -       SSH-2.0-Go      -
1639061356.758108       CbdgAt4if8TzlJen15      2001:41d0::1234:5678    -       PassiveRecon::TCP_HONEYPOT_HIT  tcp/23874       (empty) -
1639061356.758108       CbdgAt4if8TzlJen15      2001:41d0::1234:5678    -       PassiveRecon::TCP_HONEYPOT_HIT  tcp/23874       SSH-2.0-PUTTY\x0d\x0a   -
1639061356.758108       CbdgAt4if8TzlJen15      2001:41d0::1234:5678    -       PassiveRecon::SSH_CLIENT        -       SSH-2.0-PUTTY   -
1639061357.781475       CLqqrc2RYzBTPi6T59      2001:41d0::1234:5678    -       PassiveRecon::TCP_HONEYPOT_HIT  tcp/50000       (empty) -

p-l- commented 2 years ago

And here is an example of p0f (v3) output, also generated by Masscanned tests:

[2021/12/13 13:14:36] mod=syn|cli=192.0.0.0/20|srv=192.0.0.1/1152|subj=cli|os=???|dist=0|params=none|raw_sig=4:64+0:0:0:8192,0:::0
[2021/12/13 13:14:36] mod=syn|cli=192.0.0.0/20|srv=192.0.0.1/2003|subj=cli|os=???|dist=0|params=none|raw_sig=4:64+0:0:0:8192,0:::0
[2021/12/13 13:14:36] mod=syn|cli=192.0.0.0/20|srv=192.0.0.1/2193|subj=cli|os=???|dist=0|params=none|raw_sig=4:64+0:0:0:8192,0:::0
[2021/12/13 13:14:36] mod=syn|cli=192.0.0.0/20|srv=192.0.0.1/3709|subj=cli|os=???|dist=0|params=none|raw_sig=4:64+0:0:0:8192,0:::0
[2021/12/13 13:14:36] mod=syn|cli=192.0.0.0/20|srv=192.0.0.1/4054|subj=cli|os=???|dist=0|params=none|raw_sig=4:64+0:0:0:8192,0:::0
[2021/12/13 13:14:36] mod=syn|cli=192.0.0.0/20|srv=192.0.0.1/6605|subj=cli|os=???|dist=0|params=none|raw_sig=4:64+0:0:0:8192,0:::0
[2021/12/13 13:14:36] mod=syn|cli=192.0.0.0/20|srv=192.0.0.1/6737|subj=cli|os=???|dist=0|params=none|raw_sig=4:64+0:0:0:8192,0:::0
[2021/12/13 13:14:36] mod=syn|cli=192.0.0.0/20|srv=192.0.0.1/6875|subj=cli|os=???|dist=0|params=none|raw_sig=4:64+0:0:0:8192,0:::0
[2021/12/13 13:14:36] mod=syn|cli=192.0.0.0/20|srv=192.0.0.1/7320|subj=cli|os=???|dist=0|params=none|raw_sig=4:64+0:0:0:8192,0:::0
[2021/12/13 13:14:36] mod=syn|cli=192.0.0.0/20|srv=192.0.0.1/8898|subj=cli|os=???|dist=0|params=none|raw_sig=4:64+0:0:0:8192,0:::0
[2021/12/13 13:14:36] mod=syn|cli=192.0.0.0/20|srv=192.0.0.1/9513|subj=cli|os=???|dist=0|params=none|raw_sig=4:64+0:0:0:8192,0:::0
[2021/12/13 13:14:36] mod=syn|cli=192.0.0.0/20|srv=192.0.0.1/9738|subj=cli|os=???|dist=0|params=none|raw_sig=4:64+0:0:0:8192,0:::0
[2021/12/13 13:14:36] mod=syn|cli=192.0.0.0/20|srv=192.0.0.1/10623|subj=cli|os=???|dist=0|params=none|raw_sig=4:64+0:0:0:8192,0:::0
[2021/12/13 13:14:36] mod=syn|cli=192.0.0.0/20|srv=192.0.0.1/10723|subj=cli|os=???|dist=0|params=none|raw_sig=4:64+0:0:0:8192,0:::0
[2021/12/13 13:14:36] mod=syn|cli=192.0.0.0/20|srv=192.0.0.1/11253|subj=cli|os=???|dist=0|params=none|raw_sig=4:64+0:0:0:8192,0:::0
[2021/12/13 13:14:36] mod=syn|cli=192.0.0.0/20|srv=192.0.0.1/12125|subj=cli|os=???|dist=0|params=none|raw_sig=4:64+0:0:0:8192,0:::0
[2021/12/13 13:14:36] mod=syn|cli=192.0.0.0/20|srv=192.0.0.1/12189|subj=cli|os=???|dist=0|params=none|raw_sig=4:64+0:0:0:8192,0:::0
[2021/12/13 13:14:36] mod=syn|cli=192.0.0.0/20|srv=192.0.0.1/12873|subj=cli|os=???|dist=0|params=none|raw_sig=4:64+0:0:0:8192,0:::0
[2021/12/13 13:14:36] mod=syn|cli=192.0.0.0/20|srv=192.0.0.1/14648|subj=cli|os=???|dist=0|params=none|raw_sig=4:64+0:0:0:8192,0:::0
[2021/12/13 13:14:36] mod=syn|cli=192.0.0.0/20|srv=192.0.0.1/14659|subj=cli|os=???|dist=0|params=none|raw_sig=4:64+0:0:0:8192,0:::0
[2021/12/13 13:14:36] mod=syn|cli=192.0.0.0/20|srv=192.0.0.1/16242|subj=cli|os=???|dist=0|params=none|raw_sig=4:64+0:0:0:8192,0:::0
[2021/12/13 13:14:36] mod=syn|cli=192.0.0.0/20|srv=192.0.0.1/16243|subj=cli|os=???|dist=0|params=none|raw_sig=4:64+0:0:0:8192,0:::0
[2021/12/13 13:14:36] mod=syn|cli=192.0.0.0/20|srv=192.0.0.1/17209|subj=cli|os=???|dist=0|params=none|raw_sig=4:64+0:0:0:8192,0:::0
[2021/12/13 13:14:36] mod=syn|cli=192.0.0.0/20|srv=192.0.0.1/17492|subj=cli|os=???|dist=0|params=none|raw_sig=4:64+0:0:0:8192,0:::0
[2021/12/13 13:14:36] mod=syn|cli=192.0.0.0/20|srv=192.0.0.1/17667|subj=cli|os=???|dist=0|params=none|raw_sig=4:64+0:0:0:8192,0:::0
[2021/12/13 13:14:36] mod=syn|cli=192.0.0.0/20|srv=192.0.0.1/17838|subj=cli|os=???|dist=0|params=none|raw_sig=4:64+0:0:0:8192,0:::0
[2021/12/13 13:14:36] mod=syn|cli=192.0.0.0/20|srv=192.0.0.1/18081|subj=cli|os=???|dist=0|params=none|raw_sig=4:64+0:0:0:8192,0:::0
[2021/12/13 13:14:36] mod=syn|cli=192.0.0.0/20|srv=192.0.0.1/18682|subj=cli|os=???|dist=0|params=none|raw_sig=4:64+0:0:0:8192,0:::0
[2021/12/13 13:14:36] mod=syn|cli=192.0.0.0/20|srv=192.0.0.1/18790|subj=cli|os=???|dist=0|params=none|raw_sig=4:64+0:0:0:8192,0:::0
[2021/12/13 13:14:36] mod=syn|cli=192.0.0.0/20|srv=192.0.0.1/19124|subj=cli|os=???|dist=0|params=none|raw_sig=4:64+0:0:0:8192,0:::0
[2021/12/13 13:14:36] mod=syn|cli=192.0.0.0/20|srv=192.0.0.1/19288|subj=cli|os=???|dist=0|params=none|raw_sig=4:64+0:0:0:8192,0:::0
[2021/12/13 13:14:36] mod=syn|cli=192.0.0.0/20|srv=192.0.0.1/19558|subj=cli|os=???|dist=0|params=none|raw_sig=4:64+0:0:0:8192,0:::0
[2021/12/13 13:14:36] mod=syn|cli=192.0.0.0/20|srv=192.0.0.1/19628|subj=cli|os=???|dist=0|params=none|raw_sig=4:64+0:0:0:8192,0:::0
[2021/12/13 13:14:36] mod=syn|cli=192.0.0.0/20|srv=192.0.0.1/19789|subj=cli|os=???|dist=0|params=none|raw_sig=4:64+0:0:0:8192,0:::0
[2021/12/13 13:14:36] mod=syn|cli=192.0.0.0/20|srv=192.0.0.1/20093|subj=cli|os=???|dist=0|params=none|raw_sig=4:64+0:0:0:8192,0:::0
[2021/12/13 13:14:36] mod=syn|cli=192.0.0.0/20|srv=192.0.0.1/21014|subj=cli|os=???|dist=0|params=none|raw_sig=4:64+0:0:0:8192,0:::0
[2021/12/13 13:14:36] mod=syn|cli=192.0.0.0/20|srv=192.0.0.1/21459|subj=cli|os=???|dist=0|params=none|raw_sig=4:64+0:0:0:8192,0:::0
[2021/12/13 13:14:36] mod=syn|cli=192.0.0.0/20|srv=192.0.0.1/21740|subj=cli|os=???|dist=0|params=none|raw_sig=4:64+0:0:0:8192,0:::0
[2021/12/13 13:14:36] mod=syn|cli=192.0.0.0/20|srv=192.0.0.1/24070|subj=cli|os=???|dist=0|params=none|raw_sig=4:64+0:0:0:8192,0:::0
[2021/12/13 13:14:36] mod=syn|cli=192.0.0.0/20|srv=192.0.0.1/24312|subj=cli|os=???|dist=0|params=none|raw_sig=4:64+0:0:0:8192,0:::0
[2021/12/13 13:14:36] mod=syn|cli=192.0.0.0/20|srv=192.0.0.1/24576|subj=cli|os=???|dist=0|params=none|raw_sig=4:64+0:0:0:8192,0:::0
[2021/12/13 13:14:36] mod=syn|cli=192.0.0.0/20|srv=192.0.0.1/26939|subj=cli|os=???|dist=0|params=none|raw_sig=4:64+0:0:0:8192,0:::0
[2021/12/13 13:14:36] mod=syn|cli=192.0.0.0/20|srv=192.0.0.1/27136|subj=cli|os=???|dist=0|params=none|raw_sig=4:64+0:0:0:8192,0:::0
[2021/12/13 13:14:36] mod=syn|cli=192.0.0.0/20|srv=192.0.0.1/27165|subj=cli|os=???|dist=0|params=none|raw_sig=4:64+0:0:0:8192,0:::0
[2021/12/13 13:14:36] mod=syn|cli=192.0.0.0/20|srv=192.0.0.1/27361|subj=cli|os=???|dist=0|params=none|raw_sig=4:64+0:0:0:8192,0:::0
[2021/12/13 13:14:36] mod=syn|cli=192.0.0.0/20|srv=192.0.0.1/29971|subj=cli|os=???|dist=0|params=none|raw_sig=4:64+0:0:0:8192,0:::0
[2021/12/13 13:14:36] mod=syn|cli=192.0.0.0/20|srv=192.0.0.1/31088|subj=cli|os=???|dist=0|params=none|raw_sig=4:64+0:0:0:8192,0:::0
[2021/12/13 13:14:36] mod=syn|cli=192.0.0.0/20|srv=192.0.0.1/33011|subj=cli|os=???|dist=0|params=none|raw_sig=4:64+0:0:0:8192,0:::0
[2021/12/13 13:14:36] mod=syn|cli=192.0.0.0/20|srv=192.0.0.1/33068|subj=cli|os=???|dist=0|params=none|raw_sig=4:64+0:0:0:8192,0:::0
[2021/12/13 13:14:36] mod=syn|cli=192.0.0.0/20|srv=192.0.0.1/34990|subj=cli|os=???|dist=0|params=none|raw_sig=4:64+0:0:0:8192,0:::0
[2021/12/13 13:14:36] mod=syn|cli=192.0.0.0/20|srv=192.0.0.1/35093|subj=cli|os=???|dist=0|params=none|raw_sig=4:64+0:0:0:8192,0:::0
[2021/12/13 13:14:36] mod=syn|cli=192.0.0.0/20|srv=192.0.0.1/35958|subj=cli|os=???|dist=0|params=none|raw_sig=4:64+0:0:0:8192,0:::0
[2021/12/13 13:14:36] mod=syn|cli=192.0.0.0/20|srv=192.0.0.1/36626|subj=cli|os=???|dist=0|params=none|raw_sig=4:64+0:0:0:8192,0:::0
[2021/12/13 13:14:36] mod=syn|cli=192.0.0.0/20|srv=192.0.0.1/36789|subj=cli|os=???|dist=0|params=none|raw_sig=4:64+0:0:0:8192,0:::0
[2021/12/13 13:14:36] mod=syn|cli=192.0.0.0/20|srv=192.0.0.1/37130|subj=cli|os=???|dist=0|params=none|raw_sig=4:64+0:0:0:8192,0:::0
[2021/12/13 13:14:36] mod=syn|cli=192.0.0.0/20|srv=192.0.0.1/37238|subj=cli|os=???|dist=0|params=none|raw_sig=4:64+0:0:0:8192,0:::0
[2021/12/13 13:14:36] mod=syn|cli=192.0.0.0/20|srv=192.0.0.1/37256|subj=cli|os=???|dist=0|params=none|raw_sig=4:64+0:0:0:8192,0:::0
[2021/12/13 13:14:36] mod=syn|cli=192.0.0.0/20|srv=192.0.0.1/37697|subj=cli|os=???|dist=0|params=none|raw_sig=4:64+0:0:0:8192,0:::0
[2021/12/13 13:14:36] mod=syn|cli=192.0.0.0/20|srv=192.0.0.1/37890|subj=cli|os=???|dist=0|params=none|raw_sig=4:64+0:0:0:8192,0:::0
[2021/12/13 13:14:36] mod=syn|cli=192.0.0.0/20|srv=192.0.0.1/38958|subj=cli|os=???|dist=0|params=none|raw_sig=4:64+0:0:0:8192,0:::0
[2021/12/13 13:14:36] mod=syn|cli=192.0.0.0/20|srv=192.0.0.1/42131|subj=cli|os=???|dist=0|params=none|raw_sig=4:64+0:0:0:8192,0:::0
[2021/12/13 13:14:36] mod=syn|cli=192.0.0.0/20|srv=192.0.0.1/43864|subj=cli|os=???|dist=0|params=none|raw_sig=4:64+0:0:0:8192,0:::0
[2021/12/13 13:14:36] mod=syn|cli=192.0.0.0/20|srv=192.0.0.1/44420|subj=cli|os=???|dist=0|params=none|raw_sig=4:64+0:0:0:8192,0:::0
[2021/12/13 13:14:36] mod=syn|cli=192.0.0.0/20|srv=192.0.0.1/44655|subj=cli|os=???|dist=0|params=none|raw_sig=4:64+0:0:0:8192,0:::0
[2021/12/13 13:14:36] mod=syn|cli=192.0.0.0/20|srv=192.0.0.1/44868|subj=cli|os=???|dist=0|params=none|raw_sig=4:64+0:0:0:8192,0:::0
[2021/12/13 13:14:36] mod=syn|cli=192.0.0.0/20|srv=192.0.0.1/45157|subj=cli|os=???|dist=0|params=none|raw_sig=4:64+0:0:0:8192,0:::0
[2021/12/13 13:14:36] mod=syn|cli=192.0.0.0/20|srv=192.0.0.1/46213|subj=cli|os=???|dist=0|params=none|raw_sig=4:64+0:0:0:8192,0:::0
[2021/12/13 13:14:36] mod=syn|cli=192.0.0.0/20|srv=192.0.0.1/46497|subj=cli|os=???|dist=0|params=none|raw_sig=4:64+0:0:0:8192,0:::0
[2021/12/13 13:14:36] mod=syn|cli=192.0.0.0/20|srv=192.0.0.1/46955|subj=cli|os=???|dist=0|params=none|raw_sig=4:64+0:0:0:8192,0:::0
[2021/12/13 13:14:36] mod=syn|cli=192.0.0.0/20|srv=192.0.0.1/49049|subj=cli|os=???|dist=0|params=none|raw_sig=4:64+0:0:0:8192,0:::0
[2021/12/13 13:14:36] mod=syn|cli=192.0.0.0/20|srv=192.0.0.1/49067|subj=cli|os=???|dist=0|params=none|raw_sig=4:64+0:0:0:8192,0:::0
[2021/12/13 13:14:36] mod=syn|cli=192.0.0.0/20|srv=192.0.0.1/49452|subj=cli|os=???|dist=0|params=none|raw_sig=4:64+0:0:0:8192,0:::0
[2021/12/13 13:14:36] mod=syn|cli=192.0.0.0/20|srv=192.0.0.1/49480|subj=cli|os=???|dist=0|params=none|raw_sig=4:64+0:0:0:8192,0:::0
[2021/12/13 13:14:36] mod=syn|cli=192.0.0.0/20|srv=192.0.0.1/50498|subj=cli|os=???|dist=0|params=none|raw_sig=4:64+0:0:0:8192,0:::0
[2021/12/13 13:14:36] mod=syn|cli=192.0.0.0/20|srv=192.0.0.1/50945|subj=cli|os=???|dist=0|params=none|raw_sig=4:64+0:0:0:8192,0:::0
[2021/12/13 13:14:36] mod=syn|cli=192.0.0.0/20|srv=192.0.0.1/51181|subj=cli|os=???|dist=0|params=none|raw_sig=4:64+0:0:0:8192,0:::0
[2021/12/13 13:14:36] mod=syn|cli=192.0.0.0/20|srv=192.0.0.1/52890|subj=cli|os=???|dist=0|params=none|raw_sig=4:64+0:0:0:8192,0:::0
[2021/12/13 13:14:36] mod=syn|cli=192.0.0.0/20|srv=192.0.0.1/53301|subj=cli|os=???|dist=0|params=none|raw_sig=4:64+0:0:0:8192,0:::0
[2021/12/13 13:14:36] mod=syn|cli=192.0.0.0/20|srv=192.0.0.1/53407|subj=cli|os=???|dist=0|params=none|raw_sig=4:64+0:0:0:8192,0:::0
[2021/12/13 13:14:36] mod=syn|cli=192.0.0.0/20|srv=192.0.0.1/53417|subj=cli|os=???|dist=0|params=none|raw_sig=4:64+0:0:0:8192,0:::0
[2021/12/13 13:14:36] mod=syn|cli=192.0.0.0/20|srv=192.0.0.1/53980|subj=cli|os=???|dist=0|params=none|raw_sig=4:64+0:0:0:8192,0:::0
[2021/12/13 13:14:36] mod=syn|cli=192.0.0.0/20|srv=192.0.0.1/55827|subj=cli|os=???|dist=0|params=none|raw_sig=4:64+0:0:0:8192,0:::0
[2021/12/13 13:14:36] mod=syn|cli=192.0.0.0/20|srv=192.0.0.1/56483|subj=cli|os=???|dist=0|params=none|raw_sig=4:64+0:0:0:8192,0:::0
[2021/12/13 13:14:36] mod=syn|cli=192.0.0.0/20|srv=192.0.0.1/58552|subj=cli|os=???|dist=0|params=none|raw_sig=4:64+0:0:0:8192,0:::0
[2021/12/13 13:14:36] mod=syn|cli=192.0.0.0/20|srv=192.0.0.1/58713|subj=cli|os=???|dist=0|params=none|raw_sig=4:64+0:0:0:8192,0:::0
[2021/12/13 13:14:36] mod=syn|cli=192.0.0.0/20|srv=192.0.0.1/58836|subj=cli|os=???|dist=0|params=none|raw_sig=4:64+0:0:0:8192,0:::0
[2021/12/13 13:14:36] mod=syn|cli=192.0.0.0/20|srv=192.0.0.1/59362|subj=cli|os=???|dist=0|params=none|raw_sig=4:64+0:0:0:8192,0:::0
[2021/12/13 13:14:36] mod=syn|cli=192.0.0.0/20|srv=192.0.0.1/59560|subj=cli|os=???|dist=0|params=none|raw_sig=4:64+0:0:0:8192,0:::0
[2021/12/13 13:14:36] mod=syn|cli=192.0.0.0/20|srv=192.0.0.1/60534|subj=cli|os=???|dist=0|params=none|raw_sig=4:64+0:0:0:8192,0:::0
[2021/12/13 13:14:36] mod=syn|cli=192.0.0.0/20|srv=192.0.0.1/60555|subj=cli|os=???|dist=0|params=none|raw_sig=4:64+0:0:0:8192,0:::0
[2021/12/13 13:14:36] mod=syn|cli=192.0.0.0/20|srv=192.0.0.1/60660|subj=cli|os=???|dist=0|params=none|raw_sig=4:64+0:0:0:8192,0:::0
[2021/12/13 13:14:36] mod=syn|cli=192.0.0.0/20|srv=192.0.0.1/61615|subj=cli|os=???|dist=0|params=none|raw_sig=4:64+0:0:0:8192,0:::0
[2021/12/13 13:14:36] mod=syn|cli=192.0.0.0/20|srv=192.0.0.1/62402|subj=cli|os=???|dist=0|params=none|raw_sig=4:64+0:0:0:8192,0:::0
[2021/12/13 13:14:36] mod=syn|cli=192.0.0.0/20|srv=192.0.0.1/62533|subj=cli|os=???|dist=0|params=none|raw_sig=4:64+0:0:0:8192,0:::0
[2021/12/13 13:14:36] mod=syn|cli=192.0.0.0/20|srv=192.0.0.1/62941|subj=cli|os=???|dist=0|params=none|raw_sig=4:64+0:0:0:8192,0:::0
[2021/12/13 13:14:36] mod=syn|cli=192.0.0.0/20|srv=192.0.0.1/63240|subj=cli|os=???|dist=0|params=none|raw_sig=4:64+0:0:0:8192,0:::0
[2021/12/13 13:14:36] mod=syn|cli=192.0.0.0/20|srv=192.0.0.1/63339|subj=cli|os=???|dist=0|params=none|raw_sig=4:64+0:0:0:8192,0:::0
[2021/12/13 13:14:36] mod=syn|cli=192.0.0.0/20|srv=192.0.0.1/63616|subj=cli|os=???|dist=0|params=none|raw_sig=4:64+0:0:0:8192,0:::0
[2021/12/13 13:14:36] mod=syn|cli=192.0.0.0/20|srv=192.0.0.1/64380|subj=cli|os=???|dist=0|params=none|raw_sig=4:64+0:0:0:8192,0:::0
[2021/12/13 13:14:36] mod=syn|cli=192.0.0.0/20|srv=192.0.0.1/65438|subj=cli|os=???|dist=0|params=none|raw_sig=4:64+0:0:0:8192,0:::0
[2021/12/13 13:14:37] mod=syn|cli=192.0.0.0/26695|srv=192.0.0.1/445|subj=cli|os=???|dist=0|params=none|raw_sig=4:64+0:0:0:8192,0:::0
[2021/12/13 13:14:39] mod=syn|cli=2001:41d0:0:0:0:0:1234:5678/26695|srv=2001:41d0:0:0:0:0:ab32:bdb8/445|subj=cli|os=???|dist=0|params=none|raw_sig=6:64+0:0:0:8192,0:::0
[2021/12/13 13:14:40] mod=syn|cli=192.0.0.0/24592|srv=192.0.0.1/80|subj=cli|os=???|dist=0|params=none|raw_sig=4:64+0:0:0:8192,0:::0
[2021/12/13 13:14:41] mod=syn|cli=192.0.0.0/24592|srv=192.0.0.1/443|subj=cli|os=???|dist=0|params=none|raw_sig=4:64+0:0:0:8192,0:::0
[2021/12/13 13:14:42] mod=syn|cli=192.0.0.0/24592|srv=192.0.0.1/5000|subj=cli|os=???|dist=0|params=none|raw_sig=4:64+0:0:0:8192,0:::0
[2021/12/13 13:14:43] mod=syn|cli=192.0.0.0/24592|srv=192.0.0.1/53228|subj=cli|os=???|dist=0|params=none|raw_sig=4:64+0:0:0:8192,0:::0
[2021/12/13 13:14:44] mod=syn|cli=2001:41d0:0:0:0:0:1234:5678/24592|srv=2001:41d0:0:0:0:0:ab32:bdb8/80|subj=cli|os=???|dist=0|params=none|raw_sig=6:64+0:0:0:8192,0:::0
[2021/12/13 13:14:45] mod=syn|cli=2001:41d0:0:0:0:0:1234:5678/24592|srv=2001:41d0:0:0:0:0:ab32:bdb8/443|subj=cli|os=???|dist=0|params=none|raw_sig=6:64+0:0:0:8192,0:::0
[2021/12/13 13:14:46] mod=syn|cli=2001:41d0:0:0:0:0:1234:5678/24592|srv=2001:41d0:0:0:0:0:ab32:bdb8/5000|subj=cli|os=???|dist=0|params=none|raw_sig=6:64+0:0:0:8192,0:::0
[2021/12/13 13:14:47] mod=syn|cli=2001:41d0:0:0:0:0:1234:5678/24592|srv=2001:41d0:0:0:0:0:ab32:bdb8/53228|subj=cli|os=???|dist=0|params=none|raw_sig=6:64+0:0:0:8192,0:::0
[2021/12/13 13:14:48] mod=syn|cli=192.0.0.0/24592|srv=192.0.0.1/80|subj=cli|os=???|dist=0|params=none|raw_sig=4:64+0:0:0:8192,0:::0
[2021/12/13 13:14:50] mod=syn|cli=192.0.0.0/24592|srv=192.0.0.1/443|subj=cli|os=???|dist=0|params=none|raw_sig=4:64+0:0:0:8192,0:::0
[2021/12/13 13:14:51] mod=syn|cli=192.0.0.0/24592|srv=192.0.0.1/5000|subj=cli|os=???|dist=0|params=none|raw_sig=4:64+0:0:0:8192,0:::0
[2021/12/13 13:14:52] mod=syn|cli=192.0.0.0/24592|srv=192.0.0.1/53228|subj=cli|os=???|dist=0|params=none|raw_sig=4:64+0:0:0:8192,0:::0
[2021/12/13 13:14:57] mod=syn|cli=2001:41d0:0:0:0:0:1234:5678/24592|srv=2001:41d0:0:0:0:0:ab32:bdb8/80|subj=cli|os=???|dist=0|params=none|raw_sig=6:64+0:0:0:8192,0:::0
[2021/12/13 13:14:58] mod=syn|cli=2001:41d0:0:0:0:0:1234:5678/24592|srv=2001:41d0:0:0:0:0:ab32:bdb8/443|subj=cli|os=???|dist=0|params=none|raw_sig=6:64+0:0:0:8192,0:::0
[2021/12/13 13:14:59] mod=syn|cli=2001:41d0:0:0:0:0:1234:5678/24592|srv=2001:41d0:0:0:0:0:ab32:bdb8/5000|subj=cli|os=???|dist=0|params=none|raw_sig=6:64+0:0:0:8192,0:::0
[2021/12/13 13:15:00] mod=syn|cli=2001:41d0:0:0:0:0:1234:5678/24592|srv=2001:41d0:0:0:0:0:ab32:bdb8/53228|subj=cli|os=???|dist=0|params=none|raw_sig=6:64+0:0:0:8192,0:::0
[2021/12/13 13:15:05] mod=syn|cli=192.0.0.0/37183|srv=192.0.0.1/22|subj=cli|os=???|dist=0|params=none|raw_sig=4:64+0:0:0:8192,0:::0
[2021/12/13 13:15:06] mod=syn|cli=192.0.0.0/37183|srv=192.0.0.1/80|subj=cli|os=???|dist=0|params=none|raw_sig=4:64+0:0:0:8192,0:::0
[2021/12/13 13:15:07] mod=syn|cli=192.0.0.0/37183|srv=192.0.0.1/2222|subj=cli|os=???|dist=0|params=none|raw_sig=4:64+0:0:0:8192,0:::0
[2021/12/13 13:15:08] mod=syn|cli=192.0.0.0/37183|srv=192.0.0.1/2022|subj=cli|os=???|dist=0|params=none|raw_sig=4:64+0:0:0:8192,0:::0
[2021/12/13 13:15:09] mod=syn|cli=192.0.0.0/37183|srv=192.0.0.1/23874|subj=cli|os=???|dist=0|params=none|raw_sig=4:64+0:0:0:8192,0:::0
[2021/12/13 13:15:10] mod=syn|cli=192.0.0.0/37183|srv=192.0.0.1/50000|subj=cli|os=???|dist=0|params=none|raw_sig=4:64+0:0:0:8192,0:::0
[2021/12/13 13:15:11] mod=syn|cli=2001:41d0:0:0:0:0:1234:5678/37183|srv=2001:41d0:0:0:0:0:ab32:bdb8/22|subj=cli|os=???|dist=0|params=none|raw_sig=6:64+0:0:0:8192,0:::0
[2021/12/13 13:15:12] mod=syn|cli=2001:41d0:0:0:0:0:1234:5678/37183|srv=2001:41d0:0:0:0:0:ab32:bdb8/80|subj=cli|os=???|dist=0|params=none|raw_sig=6:64+0:0:0:8192,0:::0
[2021/12/13 13:15:13] mod=syn|cli=2001:41d0:0:0:0:0:1234:5678/37183|srv=2001:41d0:0:0:0:0:ab32:bdb8/2222|subj=cli|os=???|dist=0|params=none|raw_sig=6:64+0:0:0:8192,0:::0
[2021/12/13 13:15:14] mod=syn|cli=2001:41d0:0:0:0:0:1234:5678/37183|srv=2001:41d0:0:0:0:0:ab32:bdb8/2022|subj=cli|os=???|dist=0|params=none|raw_sig=6:64+0:0:0:8192,0:::0
[2021/12/13 13:15:15] mod=syn|cli=2001:41d0:0:0:0:0:1234:5678/37183|srv=2001:41d0:0:0:0:0:ab32:bdb8/23874|subj=cli|os=???|dist=0|params=none|raw_sig=6:64+0:0:0:8192,0:::0
[2021/12/13 13:15:16] mod=syn|cli=2001:41d0:0:0:0:0:1234:5678/37183|srv=2001:41d0:0:0:0:0:ab32:bdb8/50000|subj=cli|os=???|dist=0|params=none|raw_sig=6:64+0:0:0:8192,0:::0

Unactived commented 2 years ago

I think some useful features towards logs enhancements would be:

Support of "ingestible formats"

As mentioned in the opening comment, using a more universal format like JSON or logfmt allows easy redirection to broad data aggregation platforms, and better later processing of these logs.

e.g. when a key-value format like JSON is used, a process going through the logs doesn't have to think about how they're written, which column is the one with the value it needs, and that makes for clearer/shorter interaction.

Format-agnostic logging, and format selection

The codebase kind of already works like that / supports it if one adds his own Rust logger, but the tool as is does not.

The idea being to have a CLI option to select the format in which to output logs, e.g. json, logfmt, tabular.

Implementation-wise, this could be done by having the various protocol_verb (like tcp_recv) functions invoke some uinversal log function giving it the appropriate data.

Maybe through a struct with defaults (and Options for good measure?) so you can avoid telling the function things you don't have (e.g. IP addresses when logging Ethernet events) and let the log function handle it according to the format : just no key in something like JSON, an empty string in the tab-separated current format...

The universal log function would then use the chosen format to print the data.

Perfect consistency of all logs in regard to the format

In its current state the arp logging is different from all the others (eth, ipv4, ipv6, icmpv4, icmpv6, tcp, udp). As one relies on the columns to get a value and know what it is, it differs between these as a result, since it doesn't use client_info.

Configuring what to log from CLI

Currently the logger just sets to true all protocols and logs everything. One could imagine an option to unset and say, don't print events from the ethernet protocol, or some verbs like init and drop while keeping recv and send.

Log application layer formats, in the same way as other protocols

Coupled with choosing what protocols to log, I think it would just be very neat.

Include TCP/UDP ports in `recv` logs

This is a minor detail and change, it's just about calling the log function after adding that information to client_info so it can appear in it, in https://github.com/ivre/masscanned/blob/3122d4e362f4062fe73827964ad3ab020ce5ad29/src/layer_4/tcp.rs#L34-L37 and equivalent code in udp.rs

I would have made a pull request for it but I didn't know if it was deliberate ; and it's such a trivial thing I guess it fits here better than in its own issue.

Frky commented 2 years ago

Thank you for your input on the logging. Your ideas are interesting, and the logging part is definitely to be improved. Here are some comments:

Support of "ingestible formats": fully agree, and the code base was designed to be able to add more log formats later => adding a json output is clearly something we can/should do ;
Format-agnostic logging, and format selection: the idea in the current code base is to add loggers, and I think it would be easier to add a json logger than an universal logger as you suggest (but tell me if I'm wrong). Then, adding a CLI option to choose the log format (that would enable one or the other logger) could do the trick. What do you think?
Perfect consistency of all logs in regard to the format: this is a big question. So far, only arp is different from the others, but what about when we will add the application protocols? Do you have any suggestion? Because so far, I don't have any satisfying solution myself (but still I agree this is a problem) ;
Log application layer formats, in the same way as other protocols: yes, indeed, in the todo as well ;
Include TCP/UDP ports in recv logs: thanks, it was a bug indeed, fixed by PR #66.

For any of these, it is obviously open to discussion, and also to PR if you'd like to contribute to the project.

Unactived commented 2 years ago

About the consistency question for application protocols, the issue I was highlighting was basically: "when you see a value at the nth line of output, how do you know what it corresponds to?"

The issues with columns for this are:

you need to remember what the columns are (minor, can be done with UI and mostly disappears when the reader is automated rather than human)
there starts to be a lot of possible columns, as soon as each protocol has its own peculiarities in that situation you either have:
- protocol-dependent columns: kind of how arp is right now compared to others then that means you need to look at the protocol to know how to interpret the column. It can become a problem when you're trying to correlate information from different protocols, and also for a human reader who has to know what's the third column for every protocol
- one big unique ordered set of columns across all protocols, that includes the peculiarities of all then you have awfully long lines of output. Awful for a human reader but a breeze for something automated however it becomes bothersome to update that set of columns, especially if you suddenly want to add/remove columns in a protocol implemented in the past
- a protocol-agnostic field, e.g. "data", but you maybe lose structural information or have to recover it yourself with logic

What I then tried to highlight is that these issues actually all disappear once you use a key-value format like json or logfmt. Each protocol (or even line) has the columns (keys) it needs, in the order it wants, and no other.

So I'm unsure of the approach to take when using a column format like the current one, and how much it can be solved in that context without any form of user interface.

I'm personally much more interested (and biased) towards automated+processed handling of the logs, and in that regard would personally just use the key-value format as an end-user.

In regard to the current state of application layer logging, what is currently given in warn messages, I feel like "the agnostic data field" is the most promising for column-based logging, and that's actually what it already kind of looks like, given how the last columns of TCP are the TCP flag values, the last columns of ICMP are the ICMP Type and Code values and so on.

Unactived commented 2 years ago

My "universal logger" suggestion was a proposition of implementation for healthily supporting the "different possible formats" feature.

It's just meant so

once you have it set up and have already 2 or 3 formats implemented with it, it's a breeze to add another format: done at the same unique place, and any external contributor could do it
it roughly avoids developing quirks that differ between supported formats

But I might have misunderstood the code, it does seem aimed towards having separate implementations of the Logger trait which seems like a nice structure.

ivre / masscanned

Logs enhancements #3

Support of "ingestible formats"

Format-agnostic logging, and format selection

Perfect consistency of all logs in regard to the format

Configuring what to log from CLI

Log application layer formats, in the same way as other protocols

Include TCP/UDP ports in `recv` logs

ivre / masscanned

Logs enhancements #3

Support of "ingestible formats"

Format-agnostic logging, and format selection

Perfect consistency of all logs in regard to the format

Configuring what to log from CLI

Log application layer formats, in the same way as other protocols

Include TCP/UDP ports in recv logs

Include TCP/UDP ports in `recv` logs