Open shankari opened 2 years ago
This is super simple because I wanted to allow analysis experts to write new metrics using jupyter notebooks and R and whatever they want and then to also not allow very fine-grained data manipulation to avoid privacy leakage through repeated queries.
There is an summer intern who is working on revamping the images to show incorporate analysed results along with error bars and she may be using plotly which seems like the standard in the dataviz team, and also supports R and python. In that case, we may adapt this to be a plotly dashboard as well.
But for now, let's get this in production.
The main challenge with getting this into production is the storage of the generated images. Right now, they are stored in files on a shared volume between the two containers. But cloud services does not like file storage because they want all their containers to be serverless.
So they will want to map this to:
So we will have to store the images in some other shared storage. Options are:
Next steps are (in parallel):
to explore the current public dashboard, it would be great if you could also address my comments on this pending PR so I can merge it. https://github.com/e-mission/em-public-dashboard/pull/36
we are planning to switch to a plotly server after the viz changes for the "count every trip" project are done. Still thinking of deploying the current dashboard as a stopgap, need to check with cloud services about the details.
The public dashboard is currently at: https://github.com/e-mission/em-public-dashboard
It is a static dashboard - it creates images (updated daily) for the aggregate metrics for the current month It has a very simple frontend that allows people to choose the month and program and then displays the static image
It essentially provisions two docker containers: one for generating the static images (that we basically run notebooks using a cronjob to generate) and a dead simple frontend using jquery that basically displays the image.