Prepopulating relations among entities for NER annotation doesn't work

[msamogh]

Describe the bug

I'm using LabelStudio for a NER + Relations Among Entities task.

I want to pre-populate some relations to the annotation (using predictions). When I pre-populate entities, it shows up fine. However, when I try to pre-populate relations, I get the following error in my browser console:

Uncaught (in promise) Error: [mobx-state-tree] Can only store references to tree nodes or identifiers, got: 'undefined'
    at be (mobx-state-tree.module.js:3662)
    at new e (mobx-state-tree.module.js:6984)
    at t.instantiate (mobx-state-tree.module.js:7243)
    at mobx-state-tree.module.js:5734
    at mobx-state-tree.module.js:5891
    at Array.forEach (<anonymous>)
    at t.forAllProps (mobx-state-tree.module.js:5890)
    at t.initializeChildNodes (mobx-state-tree.module.js:5733)
    at new t (mobx-state-tree.module.js:1442)
    at ae (mobx-state-tree.module.js:3401)

Also, the spinner keeps spinning and nothing shows up in my browser.

This is the content of my tasks.json:

[
    {
        "id": 2,
        "data": {
            "text": "sales rep with highest sales for winkle"
        },
        "predictions": [
            {
                "result": [
                    {
                        "id": "anJaSFkP",
                        "from_name": "label",
                        "to_name": "text",
                        "type": "labels",
                        "normalization": "1P sales (Local Currency)",
                        "value": {
                            "start": 23,
                            "end": 28,
                            "text": "sales",
                            "labels": [
                                "Measure"
                            ]
                        }
                    },
                    {
                        "id": "pBUYX3sZ",
                        "from_name": "label",
                        "to_name": "text",
                        "type": "labels",
                        "normalization": "Sales Representative",
                        "value": {
                            "start": 0,
                            "end": 9,
                            "text": "sales rep",
                            "labels": [
                                "Attribute"
                            ]
                        }
                    },
                    {
                        "id": "L7o4xpaW",
                        "from_name": "label",
                        "to_name": "text",
                        "type": "labels",
                        "normalization": "Winkle~~Brand",
                        "value": {
                            "start": 33,
                            "end": 39,
                            "text": "winkle",
                            "labels": [
                                "Business Object"
                            ]
                        }
                    },
                    {
                        "id": "TV7255ne",
                        "from_name": "label",
                        "to_name": "text",
                        "type": "labels",
                        "normalization": "TOP 1",
                        "value": {
                            "start": 15,
                            "end": 22,
                            "text": "highest",
                            "labels": [
                                "Rank"
                            ]
                        }
                    },
                    /* THE RELATIONS BELOW IS THE PROBLEM */
                    {
                        "direction": "right",
                        "from_id": "TV7255ne",
                        "labels": [
                            "Rank-Attribute"
                        ],
                        "to_id": "pBUYX3sZ",
                        "type": "relation"
                    },
                    {
                        "direction": "right",
                        "from_id": "TV7255ne",
                        "labels": [
                            "Rank-Measure"
                        ],
                        "to_id": "anJaSFkP",
                        "type": "relation"
                    }
                ]
            }
        ]
    },
]

This is the content of my config.xml:

<View>

  <Relations>
    <Relation value="Period-Measure" />
    <Relation value="Measure-Condition" />
    <Relation value="Rank-Attribute" />
    <Relation value="Rank-Periodicity" />
    <Relation value="Rank-Attribute (Group)" />
    <Relation value="Rank-Periodicity (Group)" />
    <Relation value="Rank-Measure" />
  </Relations>

  <Labels name="label" toName="text">
    <Label value="Measure" />
    <Label value="Business Object" />
    <Label value="Attribute" />
    <Label value="Periodicity (Global)" />
    <Label value="Periodicity (Rank)" />
    <Label value="Temporal (Specific)" />
    <Label value="Temporal (Range)" />
    <Label value="Range (Start)" />
    <Label value="Range (End)" />
    <Label value="Condition Type" />
    <Label value="Condition Value" />
    <Label value="Rank" />
  </Labels>

  <Text name="text" value="$text" />

</View>

P.S. When I annotate the relation manually myself, the resulting JSON for that relation (in the completions folder is identical to what I have in the predictions list).

[hlomzik]

Hi! Sorry for long response, I'm trying to understand is this problem still exists, because we had a fix specifically for relations deserialisation, but we got another complain right after we deployed it. I'll check it soon and come back with solution.

[msamogh]

Sure, no problem! Thank you very much!

[msamogh]

Any update on this, @hlomzik?

[msamogh]

@hlomzik Any update?

[paulakeen]

Hi everyone, I'm working "exactly" in the same direction, but have not started yet to create the "predictions". I'll be watching this thread with high interest and feed-back you on my own progress, if any.

Many thanks for your work and input.

[makseq]

@msamogh @paulakeen I've just created a new project with your label config and imported your task (but there is incorrect comma at the end). And everything is work fine. I've used release 0.7.3.

[makseq]

[makseq]

I hope this problem is fixed in https://github.com/heartexlabs/label-studio/releases/tag/v0.7.4. Feel free to open this issue if it is not.

[anja-wo]

Hi @makseq,

I have the same issue with prepopulating relations among labels.

This is the content of my task.json:

{"602": {
  "data": {
    "text": "A 41-year-old African-American female with a past medical history of diabetes type II, hypertension, and hypercholesterolemia presents to pulmonary clinic for evaluation of shortness of breath.  She has no history of childhood asthma; however, at age 26 she was diagnosed clinically with asthma based solely on symptoms during an unrelated hospitalization."
  },
  "id": 602,
  "predictions": [
    {
      "result": [
        {
          "from_name": "label",
          "id": "00",
          "to_name": "text",
          "type": "labels",
          "value": {
            "end": 77,
            "labels": [
              "medical_condition"
            ],
            "start": 69,
            "text": "diabetes"
          }
        },
        {
          "from_name": "label",
          "id": "01",
          "to_name": "text",
          "type": "labels",
          "value": {
            "end": 99,
            "labels": [
              "medical_condition"
            ],
            "start": 87,
            "text": "hypertension"
          }
        },
        {
          "from_name": "label",
          "id": "02",
          "to_name": "text",
          "type": "labels",
          "value": {
            "end": 192,
            "labels": [
              "medical_condition"
            ],
            "start": 173,
            "text": "shortness of breath"
          }
        },
        {
          "from_name": "label",
          "id": "03",
          "to_name": "text",
          "type": "labels",
          "value": {
            "end": 233,
            "labels": [
              "medical_condition"
            ],
            "start": 217,
            "text": "childhood asthma"
          }
        },
        {
          "from_name": "label",
          "id": "04",
          "to_name": "text",
          "type": "labels",
          "value": {
            "end": 253,
            "labels": [
              "age"
            ],
            "start": 247,
            "text": "age 26"
          }
        },
        {
          "from_name": "label",
          "id": "05",
          "to_name": "text",
          "type": "labels",
          "value": {
            "end": 294,
            "labels": [
              "medical_condition"
            ],
            "start": 288,
            "text": "asthma"
          }
        },
        {
          "direction": "right",
          "from_id": "03",
          "labels": [
            "has_age"
          ],
          "to_id": "04",
          "type": "relation"
        }
      ]
    }
  ]
}}

And this is the content of my config.xml:

<View>
  <Relations>
    <Relation value="has_date_time" />
    <Relation value="has_age" />
    <Relation value="has_severity" />
    <Relation value="has_frequency" />
    <Relation value="has_dosage" />
  </Relations>

  <Labels name="label" toName="text">
    <Label value="TEMPORAL" background="red"/>
    <Label value="age" background="darkorange"/>
    <Label value="medical_condition" background="orange"/>
    <Label value="severity" background="green"/>
    <Label value="frequency" background="darkblue"/>
    <Label value="treatment_medication" background="blue"/>
    <Label value="dosage" background="purple"/>
  </Labels>

  <Text name="text" value="$text"/>
</View>

Including predictions for entities works perfectly fine. Including the relations does not work properly however. It is added as a predicted relation (see below on the right). The text does not show up in the UI anymore though and the following error shows up instead:

And it is the same for me: When I annotate the relation manually myself, the resulting JSON for that relation (in the completions folder is identical to what I have in the predictions list).

[makseq]

@anja-wo Sorry for a long answer. What LS version do you use?

[shehryar-malik]

@makseq I am facing the exact same problem as @anja-wo. The entities prediction work fine. But when I add relations, I get the 'TypeError: Cannot read property 'traverseTree' of null' on labelstudio. I'm using version 0.8.2.

[MNMaqsood]

@makseq I'm also facing the same problem as that of @shehryar-malik and @anja-wo . I am using 0.8.2 version.

[makseq]

@hlomzik Could you check this, please?

[jrubensteinsp]

I have the same issue. Can someone please advise when this will be resolved?

[makseq]

@jrubensteinsp Thanks for your report, we are working on it. I'll ping here, when we finish.

[jrubensteinsp]

Thanks Max. Do you have a RC we can test? Do you expect it in 0.8.3 or 0.9.0 and what's your best guess on an ETA?

Thanks Jeff

On Sun, Jan 10, 2021 at 6:02 PM Max notifications@github.com wrote:

@jrubensteinsp https://github.com/jrubensteinsp Thanks for your report, we are working on it. I'll ping here, when we finish.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/heartexlabs/label-studio/issues/323#issuecomment-757559668, or unsubscribe https://github.com/notifications/unsubscribe-auth/AB63H7HMF2EAGYDVRDSUBBLSZIWZXANCNFSM4OARSJ3A .

--

Jeffrey D. Rubenstein President & CEO https://www.linkedin.com/in/rubensteinjeff

[makseq]

I hope it will be in 0.9.0 on this week :slightly_smiling_face: