Proof of progress reviewer

[stackatron]

What is the problem you are seeing? Please describe. Measuring active users is proving difficult. PBC needs more time and resources to tackle this challenge. Meanwhile we want to find some signal that indicates an app quality is improving.

How is this problem misaligned with goals of app mining? In theory, each reviewer should measure a spectrum of quality or success, not binary attributes. App Mining is stronger if we can measure true quality improvements.

What is the explicit recommendation you’re looking to propose?

• Proof of progress reviewer
• App builders would report release notes into some central location each month.
• The reviewer will review each set of notes, and attempt to verify the new features, fixes, or improvements. The reviewer will qualitatively judge the value these improvements to the app use case and users. If there are no release notes, or the notes are false, the app would receive a -1 score. The apps that have made confirmed progress will get a score and corresponding z-score rank (the exact criteria or range of the score is TBD).

Describe your long term considerations in proposing this change. Please include the ways you can predict this recommendation could go wrong and possible ways mitigate.

• Apps could dispute the findings of the reviewer. Although, I would claim it is a net positive if apps need to state how they are improving and those improvements are meaningful, evident, and easily realized in the app by a user. In other words, not all app improvements are equal and we should recognize that.
• The reviewer could get swamped by the amount of labor. Yes, this reviewer would need to hire interns or something to help with testing. And maybe only test the first, or maybe first-third, claim made by each app.

Additional context

• NIL could be this reviewer they are checking each app monthly anyhow.
• This idea was inspired by @friedger thanks!

cc @larrysalibra

Just an idea, looking for feedback.

[cuevasm]

Related for those curious: https://github.com/blockstack/app-mining/issues/132

[friedger]

The updates should be stored on a decentralized user-owned storage ;-)

On Fri, 2 Aug 2019, 20:05 Mitchell Cuevas, notifications@github.com wrote:

Related for those curious: #132 https://github.com/blockstack/app-mining/issues/132

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/blockstack/app-mining/issues/138?email_source=notifications&email_token=AALBYWJSEEZEL7273A5UFCLQCRZPXA5CNFSM4II732V2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD3OOYSY#issuecomment-517794891, or mute the thread https://github.com/notifications/unsubscribe-auth/AALBYWOLPGGGD7IVGOSGLHDQCRZPXANCNFSM4II732VQ .

[friedger]

This could be a dimension in #147

[stackatron]

Starting a new thread on a similar idea: https://github.com/blockstack/app-mining/issues/174