Right now, the only way a user's tasks and labels can be marked as lower quality is by marking the entire user as a "low quality user." This presents the issue that users whose labels get better over time will still be marked as low quality, making all of their new good labels useless, or being marked as high quality and their bad labels will be used when they shouldn't be. This also limits the options we have for marking user audits as either high quality or low quality.
The addition of task flags allows an Admin to manually derive when a user's tasks improve in quality and mark older tasks as low quality, meaning a user can continue making meaningful contributions even if their earlier performance was not up to par. New flags, "incomplete" and "stale", were also incorporated, which increase the options for managing user tasks by having different effects on the task's re-audit, validation, clustering, and leaderboard effects. The effects and differences of each flag listed below:
Flag Name
Re-audit?
Include in Clustering
Validate
Include in Dashboard
Include in Leaderboard
Notes
Incomplete
✅
✅
✅
✅
❌
The labels placed on the street were good, but a lot of labels were missed (false negatives)
Low Quality
✅
❌
❌
✅
❌
Street was otherwise audited poorly
Stale
✅
❌
❌
✅
✅
To be applied automatically in the future to old data, but can be applied manually now.
Before/After screenshots (if applicable)
Admin User GSV Label View:
Admin User Page (under "General Info and Stats")
Testing instructions
Sign in as an Admin account. Go to the Admin dashboard, then the "Users" tab on the left.
Select a user, preferably the top user with a lot of audits.
Scroll down to the "Set Flags by Date" section.
Click either of the "Set Flags" buttons. This will apply the specified flag to all tasks started before the current date (so all tasks).
Scroll up to the label map. Click on a label and scroll down to the "Manually Change Task Flags" section.
Check that the button for the flag(s) you modified in step 4 are red, indicating they are applied.
Click the buttons to manually change the flag of the task that the label is in.
Things to check before submitting the PR
[x] I've written a descriptive PR title.
[x] I've added/updated comments for large or confusing blocks of code.
[x] I've included before/after screenshots above.
[x] I've asked for and included translations for any user facing text that was added or modified.
[x] I've updated any logging. Clicks, keyboard presses, and other user interactions should be logged. If you're not sure how (or if you need to update the logging), ask Mikey. Then make sure the documentation on this wiki page is up to date for the logs you added/updated.
Resolves #2508
Right now, the only way a user's tasks and labels can be marked as lower quality is by marking the entire user as a "low quality user." This presents the issue that users whose labels get better over time will still be marked as low quality, making all of their new good labels useless, or being marked as high quality and their bad labels will be used when they shouldn't be. This also limits the options we have for marking user audits as either high quality or low quality.
The addition of task flags allows an Admin to manually derive when a user's tasks improve in quality and mark older tasks as low quality, meaning a user can continue making meaningful contributions even if their earlier performance was not up to par. New flags, "incomplete" and "stale", were also incorporated, which increase the options for managing user tasks by having different effects on the task's re-audit, validation, clustering, and leaderboard effects. The effects and differences of each flag listed below:
Before/After screenshots (if applicable)
Admin User GSV Label View:
Admin User Page (under "General Info and Stats")
Testing instructions
Things to check before submitting the PR