search

stacks-archive / app-mining

For App Mining landing page development and App Mining operations.
https://app.co/mining
MIT License
49 stars 16 forks source link

New apps can benefit from a higher number of good apps #201

Closed friedger closed 4 years ago

friedger commented 4 years ago

What is the problem you are seeing? Please describe. New apps do not have an awario score, their final score is calculated as avg(NIL, TMUI). The final score of an average app on awario (Z score = 0) with the same NIL and TMUI score is calcualated as avg(NIL, TMUI, 0). For example NIL score and TMUI score = 1, then the new app has final score 1, the average app final score 2/3. (Disregarding the memory function here)

How is this problem misaligned with goals of app mining? This boost for new apps is not documented and can be seen as unfair. In addition, new apps get a boost through the top10 new apps awards.

The current situation, invites new app publishers to boost other apps on awario social score in order to get a better score than the boosted apps. See #199. The more apps receive max score in NIL and Awario social score the better for new apps. For example a top app with awario 1 now, could be averaged to awario score 0.

The different calculation of scores for new apps and existing apps creates a diffuse result in app ranking that is difficult to understand by the public.

What is the explicit recommendation you’re looking to propose? Option A) Give new apps an awario Z score of 0 (= average) Option B) Document and promote it as a boost for new apps (including the memory function)

Describe your long term considerations in proposing this change. Please include the ways you can predict this recommendation could go wrong and possible ways mitigate. Option A) could result in less new apps because they start in the middle of the ranking for awario. To mitigate that the top10 new app reward could be increased. Option B) could result in many low quality apps that only try to get max score in the first month(s).

Additional context In the long run, the majority of apps will receive full score in NIL and Awario social count. This results in less relevance for NIL and awario social score. The remaining factors determine the ranking. See #119 and #154

Discussion about strategic suspension: #178

njordhov commented 4 years ago

Addendum: The effect is currently reverse for apps that are candidates for getting into the top ranks. For November, the top app debuted at rank 13, held down by a low top NIL score of ~0.6 having high impact (50%) and by not benefiting from being boosted by the oversized awario score. Using the strategy documented in #199 the top debuting apps could have depressed the scores of high ranking apps and gotten into rank #4 simply by posting the names of all apps on all five social media channels.

friedger commented 4 years ago

Not filling the missing awario score with data is against the recommendations of the game theory paper. It is suggested to use -1 in the case of democracy earth. I think for awario 0 would be a good value.

Alternatively, new apps could go through a dry run period of one month and compete for the top10 newest rewards using the NIL and TMUI only. In the seond month, the new apps will join the existing apps in the normal evaluation process.

hstove commented 4 years ago

I don't want to give new apps a zero, because that will hurt many apps' scores, and is not fair given that the result is that we simply don't have enough data yet. It's different than Democracy Earth, because those apps could have received votes, but no one voted on them. In this case, it's just not possible to have data for new apps.

This is another scenario where I'd recommend against hastily making changes for some behavior that could happen.

hstove commented 4 years ago

I'm going to close this, and we can continue discussion in #199 , which is basically the same topic.