m-lab / etl

M-Lab ingestion pipeline
Apache License 2.0
22 stars 7 forks source link

Migrate ndt5 to use standard columns #1045

Closed stephen-soltesz closed 2 years ago

stephen-soltesz commented 2 years ago

This change updates the ndt5 parser to support standard columns for the v2 gardener system.

Because the ndt5 result can include both S2C (download) and C2S (upload) but there is only a single summary record, this changes the parser to write one row per direction. The result of this decision is that the ndt5 rows will be organized more similarly to how ndt7 rows are collected and parsed, making down stream queries/filters and the unified tables easier to manage.

This change is Reviewable

coveralls commented 2 years ago

Pull Request Test Coverage Report for Build 7196


Changes Missing Coverage Covered Lines Changed/Added Lines %
parser/ndt5_result.go 74 95 77.89%
<!-- Total: 75 96 78.13% -->
Totals Coverage Status
Change from base Build 7189: 0.3%
Covered Lines: 3906
Relevant Lines: 6057

💛 - Coveralls
stephen-soltesz commented 2 years ago

Part of: