nathanielrindlaub
commented
2 years ago I built out a crude version of this (currently just displaying the returned JSON "stats" payload, with the following:
- imageCount - total # of images included in filters & in calculations
- reviewedCount - breakdown of # of images that have been reviewed / still need review
- reviewerList - a list of all the users who have edited images, and the number of images they have edited
- labelList - breakdown of all the validated objects (# skunks, foxes, empties, etc)
- multipleReviewerCount - number of images that have been edited by more than one user. Not that useful I don't think - I was just curious.
Next steps are to display these data in a more polished dashboar format (table, bar charts, etc.), and consider whether there are other things that might be interesting to show. I feel like there's opportunity to make the labelList more interesting...
We could possibly show total number of ML predictions for each label so you can compare roughly how well it performs against the validated label counts (i.e., there were 5k ML predictions for "skunks", but only 3k validated ones). Of course, that's a very imperfect gauge of model accuracy, because an invalidated false positive will be erased by a corrected false negative somewhere else.
For all the validated top labels, we could show the % that were ML predicted & simply validated vs. manually created? That might be more interesting.
Could also maybe show counts for the labels that have yet to be validated (objects still need review).
I think it would be pretty straight forward and useful to have an option to pull stats about a particular set of filtered images. Maybe we put a button next to the image count that triggers a modal to open with some analytics and simple data visualizations. Useful ones include:
We would need to create query handlers for all of these on the API side.