HumanSignal / label-studio

Label Studio is a multi-type data labeling and annotation tool with standardized output format
https://labelstud.io
Apache License 2.0
19.05k stars 2.37k forks source link

interactive-annotating endpoint called on submit #6312

Open cgrtrifork opened 1 month ago

cgrtrifork commented 1 month ago

Describe the bug The interactive annotations are called when 'Submit' or 'Update' buttons are clicked, generating duplicate annotations with Grounding SAM model.

I have a Grounding SAM model that is connected to an image annotation project. On the labeling interface I have defined the following prompt:

<TextArea name="prompt" value="object-to-find" placeholder="prompt" toName="image" editable="false" rows="1" maxSubmissions="1" showSubmitButton="true"/>

I have enabled auto-annotations and auto-accept suggestions on the labeling tasks.

I am able to use it normally: as part or my annotation workflow, writing to the prompt and clicking "Add" button calls the interactive-annotating endpoint, which results in objects being annotated with masks.

The problem is, once the annotation for the image is finished and I click on 'Submit' then the interactive annotation is run again and I end up with duplicate annotations.

To Reproduce Steps to reproduce the behavior:

  1. Set up the Grounding SAM ML backend and connect it to an image annotation project.
  2. Add the prompt and BrushLabels as specified in the Grounding SAM example.
  3. Open an image annotation task.
  4. Enable auto-annotation and auto-accept annotations on the tasks.
  5. Test that the interactive annotation works well when clicking 'Add' for a prompt, predictions are generated.
  6. Click on 'Submit'. The duplicate predictions appear. The prompt is also duplicated.

Expected behavior When clicking the 'Submit' or 'Update' button the interactive-annotation endpoint shouldn't be called on the connected ML backend since the annotation is supposedly already finished by the time 'Submit' is clicked. Otherwise duplicate annotations are generated.

Screenshots image In the screenshot above we can see that "Object-to-find" prompt has been duplicated. A new unnamed annotation has been added with 0.91 confidence due to the request for interactive annotation.

Environment (please complete the following information):

{ "release": "1.13.1", "label-studio-os-package": { "version": "1.13.1", "short_version": "1.13", "latest_version_from_pypi": "1.13.1", "latest_version_upload_time": "2024-08-20T17:38:07", "current_version_is_outdated": false }, "label-studio-os-backend": { "message": "ci: Build frontend", "commit": "d9b816a3781ee806bba50c8c5589fcdb8555a277", "date": "2024/08/19 19:14:23", "branch": "", "version": "1.13.1+0.gd9b816a" }, "label-studio-frontend": { "message": "fix: LEAP-762: Fix offsets in domManager for emojis (#6135)", "commit": "8d8a7fce72b808d00f94cc10b49669121d186c1f", "date": "2024-08-01T03:08:02.000Z", "branch": "develop" }, "dm2": { "message": "chore: OPTIC-866: Stylus to Sass conversion LS web/libs/datamanager/sr ...", "commit": "8fbedaa6fdd00338da19c258a798b8799f0cd935", "date": "2024-07-26T15:12:41.000Z", "branch": "develop" }, "label-studio-converter": { "version": "1.0.5" } }

jombooth commented 1 month ago

/jira create

Workflow run Jira issue TRIAG-823 is created

jombooth commented 1 month ago

Hi @cgrtrifork, thanks for raising this issue and sending a PR! I'll flag this for the team.

cgrtrifork commented 1 month ago

I can add that this also happens whenever any TextArea is submitted by clicking the 'Add' button. In the screenshot I sent when adding a comment it also triggers a prediction.