search

stacks-archive / app-mining

For App Mining landing page development and App Mining operations.
https://app.co/mining
MIT License
48 stars 16 forks source link

Payouts on 1rst – extend audit period to ~15 days #116

Closed stackatron closed 5 years ago

stackatron commented 5 years ago

What is the problem you are seeing? Please describe. We typically only have 2 days of audit period to spot in errors in ranking data. I believe twice a "live error" got through this review and made it to final ranking.

How is this problem misaligned with goals of app mining? Errors in ranking and payouts are bad.

What is the explicit recommendation you’re looking to propose? Extending the audit period to ~15 days. This means ranking data will be due on the 15th each month, followed by 15 days of audit. Then payments will be made on the 1rst of the month for the previous month.

GinaAbrams commented 5 years ago

I plus one this idea. Might be nice to keep the rankings and results all in the same month, in which case we could do payouts on the last business day of the month.

stackatron commented 5 years ago

No negative feedback in app miner meeting. All PBC team supports. Moving to review.

Walterion01 commented 5 years ago

I didn't say anything on this issue, because it seems your work to audit and I don't want to limit your timeline, but I think 15 days to check results is too much and give a filling of a left-open-task and wasting time. One week should be enough, but as I said, it is just my thought and don't expect you to change anything.

GinaAbrams commented 5 years ago

I am open to a week-long audit period.

hstove commented 5 years ago

I think one week is not enough time. We have seen that TMUI often isn't able to share videos with developers until at least a few days after the audit scores are available. If we only had a week-long audit, then developers would only have a day or two to review their TMUI scores.

We really want to eliminate all possible chances for error with our audit process. Combined with other changes we're making (like an audit changelog), I think a 2 week audit period is the right approach.

Walterion01 commented 5 years ago

Glad to hear that TMUI results are auditable too. We already faced some issue with last month result calling BlackHole as "BlackHole web app", or pushing users to sign-in before downloading the app. Therefore users get confused for a while to find the path( or never in one case), all that leaded to low TMUI score. In this case, I can see the time you mentioned is right.

GinaAbrams commented 5 years ago

How about 10 days. That's longer than a week to provide ample time and allows for some buffer time between months. It would be preferred from an operational perspective.

stackatron commented 5 years ago

PBC team discussed all factors and decided that 7 days is ideal.