search

stacks-archive / app-mining

For App Mining landing page development and App Mining operations.
https://app.co/mining
MIT License
49 stars 16 forks source link

Create a checklist for audit process #115

Closed stackatron closed 4 years ago

stackatron commented 5 years ago

What is the problem you are seeing? Please describe. PBC runs some internal checks to gather data and generate rankings. And then we share with the App Miners for the audit period (~2 days before final ranking). However errors still occur from time to time.

How is this problem misaligned with goals of app mining? Mistakes in ranking data and payouts are bad.

What is the explicit recommendation you’re looking to propose? We establish a checklist of all the checks PBC and App Miners will perform before we declare that the scores/results are final and should be paid accordingly. Every time we encounter a new problem, we can update the list with check to prevent it a second time.

Describe your long term considerations in proposing this change. Please include the ways you can predict this recommendation could go wrong and possible ways mitigate.

stackatron commented 5 years ago

Extend this: Include the stuff Hank does and make this a github issue based thing.

Next step: Hank adds everything to our doc for July ranking around July 15th.

hstove commented 5 years ago

This idea is definitely going to happen, so I'm going to move this to "Done". I will create the checklist during the process of gathering and sharing audit data on the 15th of the month. I can get started, but it'll be hard to get everything together without actually going through it.

hstove commented 4 years ago

I currently have a draft for the checklist. I am a bit unsure of how much detail to put in here - it could easily become a checklist with >100 steps, at which point I think the value of a checklist is lost.

GinaAbrams commented 4 years ago

@hstove can you add this to the current ops checklist template as a next step? I think we can publish on our GitHub so anyone can see shortly after.

stackatron commented 4 years ago

Sounds like this is done moving to review.

hstove commented 4 years ago

All set, added to our checklist.