m-lab / etl-gardener

Gardener provides services for maintaining and reprocessing mlab data.
Apache License 2.0
13 stars 5 forks source link

Implementation to reprocess last 12 months of data #436

Closed cristinaleonr closed 4 months ago

cristinaleonr commented 4 months ago

This PR adds an implementation to only reprocess the last 12 months of data by default. There is an option to reprocess the full archive history for a datatype by adding the FullHistory option in the config.

The changes have been running in sandbox the past couple of days with the FullHistory option enabled for ndt7 (now removed).

Screen recording 2024-02-15 3.06.09 PM.webm

This change is Reviewable

coveralls commented 4 months ago

Pull Request Test Coverage Report for Build 3718

Details


Changes Missing Coverage Covered Lines Changed/Added Lines %
job-service/job-service.go 10 11 90.91%
<!-- Total: 10 11 90.91% -->
Totals Coverage Status
Change from base Build 3709: 0.1%
Covered Lines: 1268
Relevant Lines: 1617

💛 - Coveralls