search

swb-ief / etl-pipeline

The Covid Lens
1 stars 10 forks source link

Svd09 patch r proc 3 #141

Closed rivera-lanasm closed 3 years ago

rivera-lanasm commented 3 years ago

This branch contains the Luigi tasks associated with calculating RT and DT (only for Mumbai currently) using input data from S3.

@svd09 the R_scripts/doub_time_test.R script contains a different implementation of the DT code. I believe this is equivalent to the implementation in dt_script.R, but I want to check with you.

I will create another branch based on this one, wherein I will complete implementation of RT district filtering (critical cities), implementing DT for all districts, and then implementing RT and DT for Mumbai wards as well.

Currently the process outputs to S3 path: the-covid-lens/Phase2_DT.csv and the-covid-lens/Phase2_RT.csv. I am wondering if it could make more sense to output distinct RT and DT files for each district, although I'm not sure what the dashboard currently expects to ingest.

svd09 commented 3 years ago

Thanks. For Rt that is fine. For DT we are still debating which DT function to use. There is a python DT function that we may use. So, would suggest doing that only for Rt right now.