Suspending an app restarts scoring without past scores as baggage

[njordhov]

What is the problem you are seeing? Please describe.

Suspending an app from app mining purges past scores from affecting the score. It opens up for gaming app mining by suspending an app to restart scoring without past scores as baggage.

How is this problem misaligned with goals of app mining?

Apps suspending participation in app mining to gain an advantage will be disruptive to the program and lead to administrative overhead.

What is the explicit recommendation you’re looking to propose?

There are multiple ways to close this loophole. I recommend changing the calculation of the Final score by using the Average score from the last round for Score last round (which is 25% of the Final score). This will make the score only affected by the previous month rather than all past scores.

Describe your long term considerations in proposing this change. Please include the ways you can predict this recommendation could go wrong and possible ways mitigate.

It will make the final score more dynamic and quicker to respond to changes.

Additional context

Multiple apps have restarted scores in the November 2019 audit. However, most are in low ranking apps, with no evidence of restarting scores for tactical advantage. The highest ranking app to have restarted score is ranked at 23, but would have been ranked at around 28 if its past score from September were used in the calculation.

[friedger]

Using the Average score would destroy the nice s-curve that game theorists have decided as a good solution.

The calculation could be changed to use the last final score where the app was not suspended instead of the final score of the last month/round.

[hstove]

You recommended this:

There are multiple ways to close this loophole. I recommend changing the calculation of the Final score by using the Average score from the last round for Score last round (which is 25% of the Final score). This will make the score only affected by the previous month rather than all past scores.

That has nothing to do with apps not having a previous score if they skip a month - you're just asking to change the memory function. The way to address this would be to find the last month this app had any score.

[njordhov]

That has nothing to do with apps not having a previous score if they skip a month

The suggested change to the memory function removes a reason to suspend the app for a month. With this change, whether or not somebody suspends an app for a month, scores older than from the past month will not affect their final score.

The way to address this would be to find the last month this app had any score.

That's a can of worms. The range of final scores varies over time, opening up for strategic suspension of apps to reenter when their previous score is favorable.

[hstove]

I think I'm a bit confused because I don't see how your proposed change to the memory function addresses this issue. If we changed the memory function, and then you skipped a month, you'd still be getting rid of the 'baggage' of the previous month, right?

I think your proposal to change the memory function is potentially not a bad idea. Maybe create a new proposal around that? Because I think your proposed change is about more than this "loophole" - its more about allowing all apps to get rid of their "baggage" over time.