search

CovidToday / backend

Code for statistical methods which estimates outbreak indicators at Covid Today.
https://www.covidtoday.in/
GNU General Public License v3.0
13 stars 13 forks source link

Automation of daily code run #26

Open MGMD-96 opened 4 years ago

darmis007 commented 4 years ago

can somebody describe what has to be done in this issue?

MGMD-96 commented 4 years ago

We update our data once daily, preferably as soon as latest whole-day data is available in the source API.

This update cycle involves running a python script(s) which gives the output json(s). The frontend of website is connected to these latest jsons through their raw github urls. These jsons lie in our Backend repo.

Earlier the scripts were multiple and it was all messy but now, we have worked to produce a single python script which gives all the required output files. Also the Reproduction number script involved some loops in R, and also took like 3 hours to run. Now we are changing our method altogether, which is only python and runs within minutes. So we are GO for automation.

Now the thing is to automate this daily code run. That is, the said python scripts (one each for states and districts) automatically run on a virtual machine at a preset time once every day. The output jsons need to go into the Backend repo so that website can pickup the latest data (we can change this if smth better is available).

MGMD-96 commented 4 years ago

Since I am a newbie at coding stuff, I welcome any and all suggestions to automate this. Let us know if you have something to revamp the entire flow which makes it more efficient or something like that.

@nightmare-adi @PratikMandlecha can pitch in and help out.