search

stacks-archive / app-mining

For App Mining landing page development and App Mining operations.
https://app.co/mining
MIT License
49 stars 16 forks source link

App Reviewers can't deem apps ineligible #140

Closed xanbots closed 4 years ago

xanbots commented 4 years ago

What is the problem you are seeing? Please describe. Every month some number of apps that have been deemed eligible by Blockstack PBC, and put in "Ready" status on App.co/mining/apps, are subsequently deemed ineligible during the ranking period by one or more app reviewers. Last month this happened to 10-15 apps. This typically happens because of a bug or setup used by a reviewer where they can't authenticate.

How is this problem misaligned with goals of app mining? App creators and Blockstack PBC don't have clarity of whether or not an app is eligible until after App Reviewers have submitted their rankings. Apps should have a binary understanding of App Mining eligibility at the beginning of each month. If bugs exist in apps, or individual reviewers are unable to assess certain apps, those should be ranked towards the bottom vs. not being eligible.

What is the explicit recommendation you’re looking to propose? If apps meet all requirements for eligibility, and are deemed eligible by Blockstack PBC at the beginning of an App Mining run, they are eligible for that run. Any App Reviewer who discovers bugs or issues with judging an app will give it a score reflecting this fact, and the ranking will therefore be very low.

friedger commented 4 years ago

Will Blockstack PBC review all apps each month for eligibility?

xanbots commented 4 years ago

We already do this, and is reflected on the app.co/mining/apps page with each app's status as either "Ready" or "Not Ready".

friedger commented 4 years ago

Isn't it unfair to payout rewards to app publishers that do not care, that do not fix their bugs, .. ?

GinaAbrams commented 4 years ago

@friedger I don't think this is what that ticket is arguing for. Please start a new ticket for that topic.

PBC discussed and feel it is better to set expectations for newly enrolling app miners instead of changing the current process. Closing.

friedger commented 4 years ago

My point was that reviewers should be able to flag apps as ineligible when they can review the app. With this proposal the flagged apps could become ineligible for next month if Blockstack PBC thinks so with the information provided by the reviewers

On Tue, 20 Aug 2019, 15:47 Gina Abrams, notifications@github.com wrote:

@friedger https://github.com/friedger I don't think this is what that ticket is arguing for. Please start a new ticket for that topic.

PBC discussed and feel it is better to set expectations for newly enrolling app miners instead of changing the current process. Next steps: update docs. I will follow up with copy.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/blockstack/app-mining/issues/140?email_source=notifications&email_token=AALBYWPBRHVGR2IVVO4YB23QFPYV7A5CNFSM4IJXWGOKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD4WLCQQ#issuecomment-523022658, or mute the thread https://github.com/notifications/unsubscribe-auth/AALBYWPC6QO7JHXFBBWBV6TQFPYV7ANCNFSM4IJXWGOA .

friedger commented 4 years ago

@GinaAbrams Is the proposal accepted or rejected?

GinaAbrams commented 4 years ago

We're not proceeding with it at this time @friedger