m-lab / etl-gardener

Gardener provides services for maintaining and reprocessing mlab data.
Apache License 2.0
13 stars 5 forks source link

Update README.md #374

Closed stephen-soltesz closed 2 years ago

stephen-soltesz commented 2 years ago

This change updates the gardener README with a short overview of this component in the etl data pipeline, and provides example commands for local development, running unit tests, as well as removing (believed) obsolete operational steps (https://github.com/m-lab/etl-gardener/issues/373)


This change is Reviewable

stephen-soltesz commented 2 years ago

FYI: @SaiedKazemi you might enjoy this change. :)

SaiedKazemi commented 2 years ago

README.md, line 11 at r1 (raw file):

## Overview

The v2 date pipeline depends on the gardener for daily and historical

Should date be data?

SaiedKazemi commented 2 years ago

README.md, line 21 at r1 (raw file):

For both of these modes, gardener issues Jobs (dates) to parsers that request
them. The parsers will enuemerate all files for that date and parse each, and
reports status updates to the gardener for the Job date until all are complete.

Should reports be report?

SaiedKazemi commented 2 years ago

README.md, line 39 at r1 (raw file):


Gardener maintains a status page on a separate status port, that summarizes
recent jobs, current state, and any errors. Jobs transition through a standard

Nit - suggesting s/a/the following/

SaiedKazemi commented 2 years ago

README.md, line 43 at r1 (raw file):


* Parsing - gardener has issued a job and a parser is working on it.
* postProcessing - the parser completed parsing a job.

Nit - suggesting s/post/Post/

SaiedKazemi commented 2 years ago

@stephen-soltesz Yes, I enjoyed the change :) Thanks a lot. Made a few nitpick suggestions.

coveralls commented 2 years ago

Coverage Status

Coverage remained the same at 61.144% when pulling a816b02c06afb4e81a86dd3e07e176ba051204bf on update-readme into f2a74a18b639ba05f8f65f9fa3574f1adcf15400 on master.