search

mikemiles-dev / netflow_parser

netflow_parser
Apache License 2.0
9 stars 4 forks source link

dbg! macros in parsing IPFIX packets creates too much CPU load #25

Closed karel-un closed 8 months ago

karel-un commented 8 months ago

We are parsing hundreds to thousands IPFIX packets per second and our CPU load was too high for such packet rate. With perf I found that problem is dbg! macros in IPFIX parse_bytes:

https://github.com/mikemiles-dev/netflow_parser/blob/ad41358d0c21eeaf3c997dd8f90070a9d35412d6/src/variable_versions/ipfix.rs#L336 https://github.com/mikemiles-dev/netflow_parser/blob/ad41358d0c21eeaf3c997dd8f90070a9d35412d6/src/variable_versions/ipfix.rs#L341

Commenting out these macros resulted in about 90% less CPU load. Reason for such load is that dbg! macro treats each vec item as separate message to stderr.

To fix this dbg! macros should be replaced with log::debug! or log::trace! or at least dbg! output should be changed to one line per vec.

mikemiles-dev commented 8 months ago

Will just comment this out until I have a better logging solution.