Decrease Product Hunt score until removed, and move them to onboarding, not app reviewing

[pstan26]

What is the problem you are seeing? Please describe. As Walterion described, Product Hunt may be letting in junk accounts to upvote apps, creating noise in the app mining system. Launching on product hunt is still a good thing to do generally.

How is this problem misaligned with goals of app mining? App mining should have less and less noise in the system so it can be relied upon.

What is the explicit recommendation you’re looking to propose? Reduce the weighting of product hunt by 5% of the total every month, until it’s removed as an App reviewer. Fill the remainder with New Internet Labs score for the time being. Continue requiring product hunt launching a requirement for launching your app in app mining.

Describe your long term considerations in proposing this change. Please include the ways you can predict this recommendation could go wrong and possible ways mitigate. Presumably New Internet Labs will become more robust and new App Reviewers are introduced to replace product hunt as a differentiated app reviewer.

[dantrevino]

@pstan26 why do we need a dry run? Without PH, the scores are what they are. Its not like we're tweaking values to ensure a specific outcome.

[dantrevino]

re is no need for the PHA to recognize that (check #134 samples). These values will skew the results profoundly (as happed before), so an app like Dclouds can quickly get a very high score (1.2) this month and even if we remove PH for the next month, it will use the previous month's score to keep corrupting the results and will take months for it to spend that high score (because of averaging with last round), and they can recharge that with some sponsored news by the budget provided though these fake upvotes.

We are maybe going in the right direction here, but I think you are totally missing the meantime and the considerable impact it put on us along the way.

• Can I ask if you have a solution for that?
• Should we all buy votes this month to guard our hard work against this problem?
• I propose that we can remove at least the distinct fake votes. Doing that for just the +500 vote apps would be easy and take a couple of hours and normalize the problem. Just like the case of Awario that saved us, the significant skew happened there.

@Walterion1 my understanding is the PH will not be used, so past performance on PH wont matter either.

[Walterion01]

@dantrevino I doubt that the effect of it on the last round score will be removed. As you can check the past month's results, you can see that some apps nourish last round scores to keep up and use the fund to power up the other reviewer score as well.

[kkomaz]

• Can I ask if you have a solution for that?
• Should we all buy votes this month to guard our hard work against this problem?
• I propose that we can remove at least the distinct fake votes. Doing that for just the +500 vote apps would be easy and take a couple of hours and normalize the problem. Just like the case of Awario that saved us, the significant skew happened there.

@pstan26

Can we get a 100% confirmation nothing will be done this month of October? If so... I hate to say it... but it seems like the only solution to counteract the flagged apps is to participate in the same manner and buy votes. 🤷🏻‍♂️

[GinaAbrams]

Thanks @kkomaz and everyone in the thread for the input. Out of fairness to all in the program and the rules we shared at the beginning of the month, want to stay consistent with the scoring parameters. Based on your feedback, we'll proceed in deprecating them as a reviewer so they're no longer included in scores for November.

[kkomaz]

Thanks for the clarification @GinaAbrams

[friedger]

From the readme:

@GinaAbrams "out of fairness" you want to keep the system unfair (more money implies higher ranking)? And what about the first goal in the readme?

[wilsonbright]

@GinaAbrams and @pstan26 Please don't let down the honest app developers for this month. Please remove PH from this month. I honestly feel, two months of dry run data is sufficient. Players have attacked the ranking system and its evident. Let's not make the larger ecosystem suffer because of a few bad players.

[Walterion01]

As an example that it is still growing here (#134) are more samples and a conclusion.

[stackatron]

Sounds like we are moving forward with the change in the Nov cycle as @GinaAbrams mentioned. That means this month is the last Product Hunt month. Better to be slow and thoughtful vs. making changes days before the review starts. Moving to done. @GinaAbrams and @moxiegirl can you please update the docs as needed.

[x5engine]

wise move 👍