Text classification using `<Labels>` tag not exporting multiple annotations when `choice="multiple"` (but `<Choices>` does)

[averdones]

Describe the bug When doing Text Classification with multiple options using the tags <Labels> and <Label>, the exported annotations file doesn't show the annotated labels.

However, when I use the tags <Choices> and <Choice>, the annotated choices are correctly exported.

I am trying to use Labels instead of Choices because it looks better and also because options are filtered when used along with the tag <Filter>.

Am I using the Labels tag in an unsupported way?

To Reproduce

1. Create a new project
2. In the section Data Import, import a simple CSV file that looks like this:

text
this is an some text
this is more text
third text

3. In the section Labeling Setup, use this XML file:

<View>
  <Text name="text" value="$text"/>
  <View style="box-shadow: 2px 2px 5px #999;                padding: 20px; margin-top: 2em;                border-radius: 5px;">
    <Header value="Choose text sentiment"/>
    <Labels name="sentiment" toName="text" choice="multiple" showInLine="true">
      <Label value="Positive"/>
      <Label value="Negative"/>
      <Label value="Neutral"/>
    </Labels>
  </View>
</View>

This is the same as the default template but changing Choices and Choice for Labels and Label, and adding the option choice="multiple"

4. Annotate one example
5. Export annotations as CSV
6. The exported CSV file doesn't have the annotations (when using Choices tag, the CSV file would correctly have a column named sentiment)

Expected behavior The exported CSV file should have the multiple annotations.

Environment:

• OS: Windows 10
• Label Studio Version 1.5.0

[makseq]

Please use JSON format, CSV is not yet supported here.

[averdones]

The JSON format doesn't work either. The result field inside annotations is empty when using \<Labels>:

"result":
[]

However, when using \<Choices>, the result field works as expected:

"result":
[
    {
        "value":
        {
            "choices":
            [
                "Degenerative disc disease",
                "Gout",
                "Hemophilia"
            ]
        },
        "id": "X-2Uk9VPVe",
        "from_name": "impression",
        "to_name": "text",
        "type": "choices",
        "origin": "manual"
    }
]

After debugging this, I found that the issue is in the frontend: when clicking on Submit, the http response doesn't return anything inside result. I have tried to fix this but I am not familiar with React.

Is this a functionality expected to be working for my use case?

[makseq]

Sorry for the delay, I've just checked it with LS 1.6, it works well there. https://youtu.be/ZQholb1Xz5o

[averdones]

Thank you for taking the time to record the video.

I can replicate your behavior. However, my use case is different: I would like to do text classification (not NER) where I just click on the label and click Submit. It seems that with the <Labels> tag, it is necessary to also click somewhere on the text to make results appear.

It would be very nice to have this feature in <Labels> because it is more visually pleasing than <Choices>, but also because the <Filter> works in <Labels> but not in <Choices>. But I will stick to <Choices> for now.

Edited: Sorry, the words between <> were not showing and the message didn't make any sense.

[makseq]

@averdones could you please record a video? It will be easier to discuss with the team when there is a video.

[averdones]

Here is a video where first I show the desired behavior that doesn't work (result is empty). And after, I show how to make result return something, but with an undesired behavior (it basically replicates the actions in your video):

https://youtu.be/oJxQ2nXfpQc

Ideally, I would like the first way to work. However, I understand that it might be complicate, since <Labels> tag seems to expect an interaction with the input text. But perhaps there is a way for <Labels> to mimic <Choices>'s behavior.

[averdones]

Hi @makseq . I have been looking into this a bit more and I understand that changing the behavior of <Labels> for my use case might be difficult and probably not wanted, since <Labels> is used in many annotation tasks that require an interaction with the input text or image.

Therefore, I thought I could modify a local version of Label Studio for my project. However, I can't find the exact part of the code that I need to modify to achieve what I want.

What I would need is to force the HTTP POST method http://localhost:8080/api/tasks/2057/drafts?project=11 to happen by hardcoding any text in the result object. This method happens right after creating an annotation, and before clicking on Submit.

I believe this happens in the frontend, but I am not very familiar with React, so I am not sure what part of the code I should modify.

Could you give me some pointers (if this is even feasible)?

[hlomzik]

Hi, @averdones! You want to use classification, so we have tag specifically created for this — Choices :) Labels are absolutelly different tag, so no reason to touch them here. Your concerns is about filtering and look:

1. Filter should work with Choices also, that's a known bug, we'll fix it soon. Also you can try to use Taxonomy as a temporary workaround.

2. Look can be changed by styling, just add Style tag and play around with class names:

   <Style>
   .ant-checkbox-wrapper .ant-checkbox { display: none }
   .ant-checkbox-wrapper-checked {
     outline: 1px solid grey;
   }
   </Style>

[averdones]

Thank you @hlomzik . I think you said it better than I could. I'm interested in filtering and look.

I'll play around with styles and wait for the Filter bug to be fixed (or see if taxonomy is good for my case in the meantime).

Thank you very much.