m-lab / etl

M-Lab ingestion pipeline
Apache License 2.0
22 stars 7 forks source link

Double parser replicas to process historical archive faster with tcpinfo #1071

Closed stephen-soltesz closed 2 years ago

stephen-soltesz commented 2 years ago

The introduction of the tcpinfo v2 parser has significantly slowed processing of the v2 pipeline. This change increases the replicas from 4 to 8.


This change is Reviewable

coveralls commented 2 years ago

Pull Request Test Coverage Report for Build 7272


Files with Coverage Reduction New Missed Lines %
active/active.go 4 88.54%
<!-- Total: 4 -->
Totals Coverage Status
Change from base Build 7262: -0.07%
Covered Lines: 3968
Relevant Lines: 6084

💛 - Coveralls