Examining Zoltar API traffic by users

reichlab / forecast-repository

Codebase for Zoltar forecast repository

https://zoltardata.com/

GNU General Public License v3.0

6 stars 3 forks source link

Examining Zoltar API traffic by users #252

Open Khoale1096 opened 4 years ago

Khoale1096 commented 4 years ago

A couple of metrics to keep track (more can be added if deem necessary).

Zoltar traffic report brainstorm (google doc)

Weekly:
- How many users submitted a query?
- How many queries total?
- Median number of queries by each user?
- How many rows served in total?
- Median number of rows per query?
- Breakdown by score vs forecast queries?
Overall:
- same as above but across all time

Initial list (old?)

How many queries per day per user? Potential: Heroku throughput metrics divided by number of zoltar users?
How many rows downloaded per day?
What tables are being queried more frequently?

ayushkhd commented 4 years ago

Ran Exploratory Data Analysis on Zoltar Jobs Table to derive valuable metrics & information.

Derived the following metrics -

The types of activities running on Zoltar, sorted from most to least common
List of activities done by a particular user along with the dates they were done on
Most active users
The models that are being queried the most, along with the dates they have been queried on
Most queried models, from high to low
List of all deleted forecasts with dates

ayushkhd commented 4 years ago

Next up - working on developing a dashboard visualization for all the data gathered

matthewcornell commented 4 years ago

That all sounds good. How are you planning on creating the dashboard?

I reviewed lots of Python visualization libraries and then settled on https://vega.github.io/vega-lite/ for the forecast heatmap. I plan on using it for other graphics down the road. If you don't have a strong reason for using a different library, please consider using vega-lite. I'm happy to take you through it.

Also, once you have output that @nickreich and you like, I'll work with you to get it into Zoltar, if that's what we want. Thanks! --m

ayushkhd commented 4 years ago

Generated a report for user activities & shared with Prof Nick

matthewcornell commented 4 years ago

Would you please put a copy here, or point me to its location? Also, I'd like to see any code that you think might be useful for possibly adding to Zoltar. Thank you.

ayushkhd commented 4 years ago

@matthewcornell Sure, I'll send you the report and the code.