m-lab / etl-gardener

Gardener provides services for maintaining and reprocessing mlab data.
Apache License 2.0
13 stars 5 forks source link

Add configuration for parsing Wehe data #421

Closed stephen-soltesz closed 1 year ago

stephen-soltesz commented 1 year ago

This change adds the scamper1 and annotation datatypes to the daily parsing job for Wehe.

Note: I needed to create the datasets manually, and updated m-lab/etl/cmd/update-schema to reference wehe rather than ndt. Ultimately, the update-schema command should be moved to the etl-gardener repo and it's functionality replaced. Similar steps will be necessary in mlab-staging and mlab-oti, performed by the m-lab/etl repo deployment.

./update-schema -project mlab-sandbox -experiment wehe -datatype scamper1
./update-schema -project mlab-sandbox -experiment wehe -datatype annotation2
./update-schema -project mlab-sandbox -experiment wehe -datatype hopannotation2

This change is Reviewable

coveralls commented 1 year ago

Pull Request Test Coverage Report for Build 3637


Totals Coverage Status
Change from base Build 3628: 0.0%
Covered Lines: 1259
Relevant Lines: 1605

💛 - Coveralls
stephen-soltesz commented 1 year ago

Part of: