Open cuevasm opened 4 years ago
njordhov
commented
4 years ago For context, these are examples of sites @cuevasm says current rules would preclude him from removing:
Related issues:
cuevasm
commented
4 years ago Yes, when I've tried to move more quickly to remove clearly odd sites, I get pushback from folks because we have the standing rule that we can't make changes during a scoring period. Even for ones that seem obvious, I will get backchannel messages or enough people saying to wait on the tickets - and I can't deny that's correct, we all agreed not to make changes in period. This is exacerbated by the unclear ratification process when we do agree.
As we have seen, the period we've allowed for App Miners to adjust to new rules (i.e. not making changes in a scoring period) has also opened the door for folks to exploit the lag in decision-making and stay ahead of any changes. It's also handcuffed App Reviewers from quickly bettering the end scoring result.
A clear example is in Awario where we see a new site or two rise up each month that clearly have a wonky Reach score and/or obviously isn't a News/Blog site, but is counted as such. Unfortunately, what has happened is that since we can't adapt to these as they come up (there is an unclear decision-making process that takes at least 1 month or more), people game these holes for Reach. I'm not talking fringe cases or grey areas, I'm talking obvious stuff here.
An example this month is Upwork, @wilsonbright was kind and noble enough to report it himself and say I could remove it. However, the current rules would preclude me from making this change without a month decision period because we can't make in period changes (and the last time I tried to push one through quickly, enough people benefiting fell back on this rule and we had to wait, letting the scores suffer in the meantime.)
In summary, I think the decision-making process should be faster, of course. Specifically, I think you could accomplish this by better empowering App Reviewers or whatever plays a similar function, to enforce the spirit of their review vs. being held to super-specific policies. This wouldn't only push out bad actors easier, but would reward good ones more as well. This month, I had to tell poor @joberding that her mentions didn't count because she added a hashtag to the front of them. Now, are those clearly mentions of her apps? Yes. Can I make that change to her benefit in the scoring period? Sadly no. I've also had to disallow Black Hole vs BlackHole for @walterion01. These things seem silly, but under the current rules, there is no way to make good faith exceptions (and I can see how it could turn into a slippery slope too).
So it works both ways and I think a stronger score is generally more important than giving notice about changes, even if it gives the occasional bit of whiplash in a scoring period. By moving more quickly on changes to the scores, you can get toward an overall better ranking more quickly too, which I think benefits everyone more in the long-run.