Closed rvagg closed 1 year ago
Yes this is all super helpful.
Seems like with retrieval, there should be a one to one relationship between "successful queries" (I think this happens after the query filter) and retrieval success + failure. The only question we might need to answer is whether we need a "did not attempt" category for no attempt due to simultaneous retrieval limits. in that case success + failure + did not attempt = successful queries
It's still a little unclear what the actionable item is here. Are we talking about being able to visualize the number of filtered out items as we pass through the funnel?
I'm in agreement with successful retrieval + failed retrieval + did not attempt = successful queries.
Confirmed offline with team that we want to be able to view the number of requests in each step pre and post filter.
Coming back here after looking at #145 and thinking through what we really want to see. I think now that what we are needing in the immediate term is a view of what part of the drop-off is artificial and what is due to network conditions. So:
Additionally, we want insight into counts at each of these phases but that's not something we can put in the funnel, but we can do histograms of distributions at the phases we care about. So let's set up view.Distribution()
type metrics for:
This will all become more complicated when we do #136 of course, but I think we can still make use of most of this in roughly the same way, you just have to interpret the data slightly differently.
We have a funnel now:
But it's naive. We don't include filtering so we can't tell why retrievals are dropping off - the number at the bottom isn't a reflection of potential. So we should capture what we need to better reflect that so we can figure out where the next lowest-hanging fruit is.