search

vmware / data-annotator-for-machine-learning

Data annotator for machine learning allows you to centrally create, manage and administer annotation projects for machine learning
Apache License 2.0
60 stars 22 forks source link

Users can continue annotations even if they exceed their assigned tickets #348

Closed msharara1998 closed 1 year ago

msharara1998 commented 2 years ago

Describe the bug

Suppose a user has x tickets to annotate. If he finished x tickets, he can continue labeling normally, which is inconsistent with the logic of sharing the annotation task among many users.

Reproduction steps

  1. Create a project
  2. Assign users
  3. Login in as an annotator (that has some assigned project)
  4. start annotating until exhausting all the assigned tickets.
  5. You will see that you can continue normally and annotate data examples that maybe allocated to other users (given that some other users has not completed all his annotations).

Expected behavior

Each user, upon annotating all his assigned tickets, must see that the submit button is grayed out.

Additional context

I would be thankful for a detailed description of how to modify the code, and in which specific file, so that the behavior of the application change and conform with the expected behavior mentioned above.

lijulia commented 2 years ago

@msharara1998, thanks for your feedback. This is by design not a bug. By allowing annotators to label more provides more flexibility. If the assigned tickets per annotator is fixed, it will not be easy for new added annotators to obtain tickets. It will also block slack channel unknown user’s annotation job.

Kai-Qiao commented 1 year ago

@msharara1998 Thanks for your feedback, as it's not a bug and it's a special design to help users flexy labeling, so i will close the issue.