Increase parser replicas to 6 to expedite annotation reprocessing

m-lab / etl

M-Lab ingestion pipeline

Apache License 2.0

22 stars 7 forks source link

Increase parser replicas to 6 to expedite annotation reprocessing #1035

Closed stephen-soltesz closed 2 years ago

stephen-soltesz commented 2 years ago

Once the annotation export runs, the new archives must be reprocessed by the v2 pipeline before another run of the annotation export. During e2e testing in staging, this is a very slow e2e process. So, this change increases the parser instances to 6 to complete historical reprocessing more quickly.

This change is

coveralls commented 2 years ago

Pull Request Test Coverage Report for Build 6981

0 of 0 changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 64.217%

Totals
Change from base Build 6945:	0.0%
Covered Lines:	3844
Relevant Lines:	5986

💛 - Coveralls

stephen-soltesz commented 2 years ago

obsolete