search

stacks-archive / app-mining

For App Mining landing page development and App Mining operations.
https://app.co/mining
MIT License
49 stars 16 forks source link

Proposal: Product Hunt Consideration in Scoring #55

Closed fiatexodus closed 5 years ago

fiatexodus commented 5 years ago

In my recent forum post on February App Mining, @pstan26 said:

Seems like Product Hunt is not a terribly popular app reviewer. Let’s discuss what we should do with them in the github repo I shared.

in response to this issue we raised with Product Hunt's team scoring:

Issue #3: Patrick Stanley mentioned in the January App Mining review that Blockstack has been talking with Product Hunt to get more information than just a number for the Product Hunt (PH) Team Score. The February data contained only a number. The Stealthy PH Team Score dropped from 8.85 in December to 6.25 in January and has remained there. Why? This is confusing. Stealthy underwent considerable UI and performance changes from January to February, which are reflected by Try My UI scoring (-.904 to .493). If the PH Team Score is about the product, why didn’t it reflect this? We believe the PH Team Score should be removed from App Mining until PH provides a detailed explanation about what influences it (note that we have no objection to continuing to use the Product Hunt Community Score in App Mining–though it’s game-ability is a separate issue to consider).

I've created this issue (which relates to the earlier issue #3) to formally propose that we retain the Product Hunt Community score because it is transparent, but drop the Product Hunt Team Score from consideration in App Mining Rewards until the following two conditions are met:

  1. One of the following:
    • A feedback mechanism as added so that the numerical ratings can be understood
    • A description of what the team score actually involves (i.e. users try using the App or artists evaluate the App for it's use of innovative fonts)
  2. Each App is evaluated every review period

I believe these suggestions will deliver more value to both Blockstack and App developers by helping to improve Apps as a result of Product Hunt's reviews instead of making the situation somewhat of a mystery as it stands today.

I also believe condition number 2 above might get tricky--when there are 400 apps and 200 of them are not in development, maybe it doesn't make sense to review all of them, maybe just the first 200. However even reviewing 200 apps wouldn't scale well. Here are some solutions I can think of to this issue that might make sense:

@pstan26 @larrysalibra @jeffdomke @hstove and @friedger what do you think?

Also @pstan26, what time-frame would be reasonable for such a change to occur if we decided to go ahead with it?

larrysalibra commented 5 years ago

I'm not terribly familiar with Product Hunt's scoring methodology.

Tagging @GinaAbrams to bring her into this discussion as well.

GinaAbrams commented 5 years ago

hey @AC-FTW thanks for the ping on this, we're working on an update and will share very soon. It's top of mind and receiving attention. 🙏

GinaAbrams commented 5 years ago

Hey @AC-FTW we've removed the internal team score and are only using credible product hunt votes. Link to forum post here. I am going to close this issue soon unless there are further questions. 🙏