cvat-ai / cvat

Annotate better with CVAT, the industry-leading data engine for machine learning. Used and trusted by teams at any scale, for data of any scale.
https://cvat.ai
MIT License
12.18k stars 2.95k forks source link

A Validator Group, Support for Validation and Automation of Job Assgning for Annotators & Validators #851

Closed dev225reddys closed 2 years ago

dev225reddys commented 4 years ago
  1. Another Role/Group in the Admin Section Called "Validator" for Maintaining users who only wants to validate the Annotation.
  2. A New UI for Validators which will have Something like "Correct/Wrong" options (No access to Annotation UI), and that job will be assigned back to Annotator for Corrections.
  3. Finally, As we already have Role/Groups, Some kind of Tracker which keeps track of Annotators as well as jobs and automatically Assigns jobs to Annotators and Validators.

*2nd issued already, but for Classification of Images but not for Validation of Annotation.

nmanovic commented 2 years ago

We are going to have more advanced roles and organizations. We will not have an explicit validator. Any person who is assigned on a job, can validate the corresponding task. I will close the issue as outdated.

Viditagarwal7479 commented 5 months ago

When thinking about crowdsourcing annotations, shouldn't we review the points in the issue and decide to reopen it if necessary? Instead of Validator, we can have some other name like CrowdSourced Annotater, which might refer to a group of crowdsource annotators instead of a single person. By this, when assigning a task to CrowdSourced Annotater, all the jobs in that task are assigned to them based on an internal algorithm that decides which job is assigned to which annotator. A single job is assigned to multiple annotators; similarly, an annotator would have multiple jobs to annotate.