Show respondent *rank* in addition to score

[LeaVerou]

This is likely too much work right now, but I'll propose it in case it's actually easier than it seems.

Especially in State of HTML, scores are generally expected to be lower than state of JS and state of CSS surveys, for two reasons:

1. We prioritized some cutting edge features over established, low hanging fruit ones, as there was way too much and we had to cut something, so we err'ed on the side of cutting things where the results wouldn't make a difference in terms of getting things implemented.
2. Mini features don't allow participants to express awareness, only usage, which means a feature doesn't get counted if they've known about it but had no reason to use it.

So, a worry is that participants will be disappointed by seeing these lower ratings. Something that would mitigate this would be if they were also shown where they rank compared to other participants. This would help establish that they didn't do poorly, the whole scale is just lower.

Implementation thoughts

I assume this would be tough to implement as it requires access to the results. I wonder if a cronjob could simply store 100 buckets, with data of the type: 0% score: 300 participants 1% score: 23 participants 2% score: 12 participants ...

perhaps in a data structure of the form:

[
    {start: 1, end: 300},
    {start: 301, end: 323},
    {start: 324, end: 336},
    ...
]

This would be pretty compact (can make it even more compact by only storing counts or just end). Then we can show participants a rough measure of where they rank (as a percentile, e.g. "top 5%") quite easily.

Edit: Now that we switched to a point score, it would have to use that, not percentages.

[SachaG]

This is something we actually had before, but we disabled it exactly because of this, calculating the rank on the fly for each user was too slow since it's a complex db operation. A cronjob is a good idea for a workaround, maybe at some point…

[eric-burel]

Precisely there is a quirk in Mongo that prevents us from indexing the field needed for this operation to make it faster... However a cronjob to compute quantiles every X hours would work fine indeed (we need to define a standard way to define cronjobs in our system).

[LeaVerou]

(we need to define a standard way to define cronjobs in our system)

I thought you use Vercel? If so, it's supported already: https://vercel.com/blog/cron-jobs