Is there support for Nat event logs ( example postNATSourceIPv4Address ) ?

[shafiev]

Hello to all, Is there way to support in web interface a NSEL( NAT event logs { netflow fields like postNATSourceIPv4Address } ) ?

[naim-shafiyev-m10]

@Dona21 like this

nfdump -r /var/cache/nfdump/nfcapd.202302011250 'host 91.229.90.xx' 
Event time               Event  XEvent Proto      Src IP Addr:Port          Dst IP Addr:Port     X-Src IP Addr:Port        X-Dst IP Addr:Port   In Byte Out Byte
2023-02-01 12:52:15.818  UNKNOW  Ignore TCP        172.16.2.39:34168 ->    91.229.90.xx:80     185.233.182.yy:34168 ->    91.229.90.xx:80           0        0

[WimRooseUG]

@shafiev No there is not at this moment. We also need support for this. I looked into it, and main show stopper here is mainly nfdump itself. It's easy to define custom formats via de -o fmt option, but the output is not really structured data, hence not easy to implement a general parser in nfsen-ng that would cover all use cases. Nfdump offers csv and json output, but the fields included are hardcoded, and NSEL fields are currently not included sadly - at least not in csv, and nfdump 1.7 doesn't include them in the json output neither, even when compiled with NSEL support (although that might be a bug).

It's also possible to define formats in a nfdump.conf file, and let nfdump output comma separated data, but the field codes are not included, so nfsen-ng still needs some hacking to make it work - and then again, for this single use case only.

See https://github.com/mbolli/nfsen-ng/issues/31 also

@mbolli what's your opinion about this?

[mbolli]

@WimRooseUG as long as there is no built-in support in nfdump, I'd like to leave it as it is. PRs with an elegant solution welcome though.

[WimRooseUG]

@mbolli I have been working on it this week, and with some minor changes it's possible to show nsel fields in the web UI, but it's mainly a hack, not really something we could call an elegant solution. I'll dig into the nfdump issues further, and will file a new one regarding the missing nsel fields in the json output of nfdump 1.7. I agree that first nfdump needs some enhancements in its output behavior before implementing fixes in nfsen-ng.

[mbolli]

@WimRooseUG you could create a draft PR, maybe someone else has some insight/ideas.

[WimRooseUG]

@mbolli I can confirm nfdump 1.7 json output does include NSEL fields by default - provided that nfcapd files are in the new format. So moving to json output as default will automatically provide support for NAT event logs.

CSV output option is deprecated anyway (nfdump 1.7). From man nfdump

json       Print full record as a separate json object.
csv        Legacy .csv format - will get removed in future releases. Please use json instead.
pipe       Legacy '|' separated format - will get removed in future releases. Please use json instead.

[Igs2000]

is there a way to automatically recycle the data in the nfsen-ng like the nfsen does? like the $low_water = 90 option