m-lab / etl

M-Lab ingestion pipeline
Apache License 2.0
22 stars 7 forks source link

Add some basic summary stats for pcap #1037

Closed gfr10598 closed 2 years ago

gfr10598 commented 2 years ago

This introduces basic summary stats for pcap files. It introduces "raw" and "a" records, and fields that will hopefully be easy to reach consensus on.

It also adds a small package for decoding pcap files, as the gopacket package is not very efficient. We also have prototype implementations for decoding ethernet, ip, and tcp headers that will be added later.


This change is Reviewable

coveralls commented 2 years ago

Pull Request Test Coverage Report for Build 7099


Changes Missing Coverage Covered Lines Changed/Added Lines %
parser/pcap.go 13 21 61.9%
tcpip/tcpip.go 67 89 75.28%
headers/ip.go 72 164 43.9%
<!-- Total: 155 277 55.96% -->
Files with Coverage Reduction New Missed Lines %
active/active.go 2 90.63%
<!-- Total: 2 -->
Totals Coverage Status
Change from base Build 7057: -0.6%
Covered Lines: 3946
Relevant Lines: 6201

💛 - Coveralls
stephen-soltesz commented 2 years ago

Closing old PR - and referencing the notes from this change in https://github.com/m-lab/etl/issues/1085