search

HumanSignal / label-studio

Label Studio is a multi-type data labeling and annotation tool with standardized output format
https://labelstud.io
Apache License 2.0
18.8k stars 2.34k forks source link

Unable to load task #2299

Closed ToddMorrill closed 2 years ago

ToddMorrill commented 2 years ago

Describe the bug I'm trying to use pre-annotations (i.e. predictions) to accelerate labeling. I uploaded a single NER task and am unable to load it when I click label all tasks.

To Reproduce Steps to reproduce the behavior:

  1. Create an NER project with 'Entity' and 'Possible Entity' as the label names.
  2. Upload the following task to the project. 
    [
{
    "data": {
        "text": "1.1 Overview: accounting for business combinations\nPublication date: 28 Feb 2022   \nUS Business combinations guide 1.1\nThis chapter discusses the key characteristics of a business and identifies which transactions require the application of business combination accounting. Business combination accounting is referred to as the \u201cacquisition method\u201d in ASC 805, Business Combinations. See discussion of the acquisition method in BCG 2. Determining whether the acquisition method applies to a transaction begins with understanding whether the transaction involves the acquisition of one or more businesses and whether it is a business combination within the scope of ASC 805.\nAll transactions in which an entity obtains control of one or more businesses qualify as business combinations, as described in the FASB\u2019s Master Glossary. ASC 805-10-25-1 further establishes the principle for identifying a business combination.\nExcerpt from ASC Master Glossary\nbusiness combination: A transaction or other event in which an acquirer obtains control of one or more businesses.\nExcerpt from ASC 805-10-25-1\nAn entity shall determine whether a transaction or other event is a business combination by applying the definition in this Subtopic, which requires that the assets acquired and liabilities assumed constitute a business. If the assets acquired are not a business, the reporting entity shall account for the transaction or other event as an asset acquisition."
    },
    "predictions": [
        {
            "model_version": "spacy",
            "result": [
                {
                    "from_name": "label",
                    "to_name": "text",
                    "type": "labels",
                    "readonly": false,
                    "id": 0,
                    "value": {
                        "start": 0,
                        "end": 12,
                        "text": "1.1 Overview",
                        "labels": [
                            "Entity"
                        ],
                        "score": 1
                    }
                },
                {
                    "from_name": "label",
                    "to_name": "text",
                    "type": "labels",
                    "readonly": false,
                    "id": 1,
                    "value": {
                        "start": 29,
                        "end": 50,
                        "text": "business combinations",
                        "labels": [
                            "Entity"
                        ],
                        "score": 1
                    }
                },
                {
                    "from_name": "label",
                    "to_name": "text",
                    "type": "labels",
                    "readonly": false,
                    "id": 2,
                    "value": {
                        "start": 69,
                        "end": 108,
                        "text": "28 Feb 2022   \nUS Business combinations",
                        "labels": [
                            "Entity"
                        ],
                        "score": 1
                    }
                },
                {
                    "from_name": "label",
                    "to_name": "text",
                    "type": "labels",
                    "readonly": false,
                    "id": 3,
                    "value": {
                        "start": 119,
                        "end": 131,
                        "text": "This chapter",
                        "labels": [
                            "Entity"
                        ],
                        "score": 1
                    }
                },
                {
                    "from_name": "label",
                    "to_name": "text",
                    "type": "labels",
                    "readonly": false,
                    "id": 4,
                    "value": {
                        "start": 142,
                        "end": 165,
                        "text": "the key characteristics",
                        "labels": [
                            "Entity"
                        ],
                        "score": 1
                    }
                },
                {
                    "from_name": "label",
                    "to_name": "text",
                    "type": "labels",
                    "readonly": false,
                    "id": 5,
                    "value": {
                        "start": 169,
                        "end": 179,
                        "text": "a business",
                        "labels": [
                            "Entity"
                        ],
                        "score": 1
                    }
                },
                {
                    "from_name": "label",
                    "to_name": "text",
                    "type": "labels",
                    "readonly": false,
                    "id": 6,
                    "value": {
                        "start": 184,
                        "end": 194,
                        "text": "identifies",
                        "labels": [
                            "Entity"
                        ],
                        "score": 1
                    }
                },
                {
                    "from_name": "label",
                    "to_name": "text",
                    "type": "labels",
                    "readonly": false,
                    "id": 7,
                    "value": {
                        "start": 195,
                        "end": 213,
                        "text": "which transactions",
                        "labels": [
                            "Entity"
                        ],
                        "score": 1
                    }
                },
                {
                    "from_name": "label",
                    "to_name": "text",
                    "type": "labels",
                    "readonly": false,
                    "id": 8,
                    "value": {
                        "start": 222,
                        "end": 237,
                        "text": "the application",
                        "labels": [
                            "Entity"
                        ],
                        "score": 1
                    }
                },
                {
                    "from_name": "label",
                    "to_name": "text",
                    "type": "labels",
                    "readonly": false,
                    "id": 9,
                    "value": {
                        "start": 241,
                        "end": 272,
                        "text": "business combination accounting",
                        "labels": [
                            "Entity"
                        ],
                        "score": 1
                    }
                },
                {
                    "from_name": "label",
                    "to_name": "text",
                    "type": "labels",
                    "readonly": false,
                    "id": 10,
                    "value": {
                        "start": 274,
                        "end": 305,
                        "text": "Business combination accounting",
                        "labels": [
                            "Entity"
                        ],
                        "score": 1
                    }
                },
                {
                    "from_name": "label",
                    "to_name": "text",
                    "type": "labels",
                    "readonly": false,
                    "id": 11,
                    "value": {
                        "start": 324,
                        "end": 347,
                        "text": "the \u201cacquisition method",
                        "labels": [
                            "Entity"
                        ],
                        "score": 1
                    }
                },
                {
                    "from_name": "label",
                    "to_name": "text",
                    "type": "labels",
                    "readonly": false,
                    "id": 12,
                    "value": {
                        "start": 352,
                        "end": 355,
                        "text": "ASC",
                        "labels": [
                            "Entity"
                        ],
                        "score": 1
                    }
                },
                {
                    "from_name": "label",
                    "to_name": "text",
                    "type": "labels",
                    "readonly": false,
                    "id": 13,
                    "value": {
                        "start": 361,
                        "end": 382,
                        "text": "Business Combinations",
                        "labels": [
                            "Entity"
                        ],
                        "score": 1
                    }
                },
                {
                    "from_name": "label",
                    "to_name": "text",
                    "type": "labels",
                    "readonly": false,
                    "id": 14,
                    "value": {
                        "start": 388,
                        "end": 398,
                        "text": "discussion",
                        "labels": [
                            "Entity"
                        ],
                        "score": 1
                    }
                },
                {
                    "from_name": "label",
                    "to_name": "text",
                    "type": "labels",
                    "readonly": false,
                    "id": 15,
                    "value": {
                        "start": 402,
                        "end": 424,
                        "text": "the acquisition method",
                        "labels": [
                            "Entity"
                        ],
                        "score": 1
                    }
                },
                {
                    "from_name": "label",
                    "to_name": "text",
                    "type": "labels",
                    "readonly": false,
                    "id": 16,
                    "value": {
                        "start": 428,
                        "end": 431,
                        "text": "BCG",
                        "labels": [
                            "Entity"
                        ],
                        "score": 1
                    }
                },
                {
                    "from_name": "label",
                    "to_name": "text",
                    "type": "labels",
                    "readonly": false,
                    "id": 17,
                    "value": {
                        "start": 455,
                        "end": 477,
                        "text": "the acquisition method",
                        "labels": [
                            "Entity"
                        ],
                        "score": 1
                    }
                },
                {
                    "from_name": "label",
                    "to_name": "text",
                    "type": "labels",
                    "readonly": false,
                    "id": 18,
                    "value": {
                        "start": 489,
                        "end": 502,
                        "text": "a transaction",
                        "labels": [
                            "Entity"
                        ],
                        "score": 1
                    }
                },
                {
                    "from_name": "label",
                    "to_name": "text",
                    "type": "labels",
                    "readonly": false,
                    "id": 19,
                    "value": {
                        "start": 515,
                        "end": 528,
                        "text": "understanding",
                        "labels": [
                            "Entity"
                        ],
                        "score": 1
                    }
                },
                {
                    "from_name": "label",
                    "to_name": "text",
                    "type": "labels",
                    "readonly": false,
                    "id": 20,
                    "value": {
                        "start": 537,
                        "end": 552,
                        "text": "the transaction",
                        "labels": [
                            "Entity"
                        ],
                        "score": 1
                    }
                },
                {
                    "from_name": "label",
                    "to_name": "text",
                    "type": "labels",
                    "readonly": false,
                    "id": 21,
                    "value": {
                        "start": 562,
                        "end": 577,
                        "text": "the acquisition",
                        "labels": [
                            "Entity"
                        ],
                        "score": 1
                    }
                },
                {
                    "from_name": "label",
                    "to_name": "text",
                    "type": "labels",
                    "readonly": false,
                    "id": 22,
                    "value": {
                        "start": 581,
                        "end": 603,
                        "text": "one or more businesses",
                        "labels": [
                            "Entity"
                        ],
                        "score": 1
                    }
                },
                {
                    "from_name": "label",
                    "to_name": "text",
                    "type": "labels",
                    "readonly": false,
                    "id": 23,
                    "value": {
                        "start": 616,
                        "end": 618,
                        "text": "it",
                        "labels": [
                            "Entity"
                        ],
                        "score": 1
                    }
                },
                {
                    "from_name": "label",
                    "to_name": "text",
                    "type": "labels",
                    "readonly": false,
                    "id": 24,
                    "value": {
                        "start": 622,
                        "end": 644,
                        "text": "a business combination",
                        "labels": [
                            "Entity"
                        ],
                        "score": 1
                    }
                },
                {
                    "from_name": "label",
                    "to_name": "text",
                    "type": "labels",
                    "readonly": false,
                    "id": 25,
                    "value": {
                        "start": 652,
                        "end": 661,
                        "text": "the scope",
                        "labels": [
                            "Entity"
                        ],
                        "score": 1
                    }
                },
                {
                    "from_name": "label",
                    "to_name": "text",
                    "type": "labels",
                    "readonly": false,
                    "id": 26,
                    "value": {
                        "start": 665,
                        "end": 668,
                        "text": "ASC",
                        "labels": [
                            "Entity"
                        ],
                        "score": 1
                    }
                },
                {
                    "from_name": "label",
                    "to_name": "text",
                    "type": "labels",
                    "readonly": false,
                    "id": 27,
                    "value": {
                        "start": 674,
                        "end": 690,
                        "text": "All transactions",
                        "labels": [
                            "Entity"
                        ],
                        "score": 1
                    }
                },
                {
                    "from_name": "label",
                    "to_name": "text",
                    "type": "labels",
                    "readonly": false,
                    "id": 28,
                    "value": {
                        "start": 700,
                        "end": 709,
                        "text": "an entity",
                        "labels": [
                            "Entity"
                        ],
                        "score": 1
                    }
                },
                {
                    "from_name": "label",
                    "to_name": "text",
                    "type": "labels",
                    "readonly": false,
                    "id": 29,
                    "value": {
                        "start": 718,
                        "end": 725,
                        "text": "control",
                        "labels": [
                            "Entity"
                        ],
                        "score": 1
                    }
                },
                {
                    "from_name": "label",
                    "to_name": "text",
                    "type": "labels",
                    "readonly": false,
                    "id": 30,
                    "value": {
                        "start": 729,
                        "end": 751,
                        "text": "one or more businesses",
                        "labels": [
                            "Entity"
                        ],
                        "score": 1
                    }
                },
                {
                    "from_name": "label",
                    "to_name": "text",
                    "type": "labels",
                    "readonly": false,
                    "id": 31,
                    "value": {
                        "start": 763,
                        "end": 784,
                        "text": "business combinations",
                        "labels": [
                            "Entity"
                        ],
                        "score": 1
                    }
                },
                {
                    "from_name": "label",
                    "to_name": "text",
                    "type": "labels",
                    "readonly": false,
                    "id": 32,
                    "value": {
                        "start": 802,
                        "end": 828,
                        "text": "the FASB\u2019s Master Glossary",
                        "labels": [
                            "Entity"
                        ],
                        "score": 1
                    }
                },
                {
                    "from_name": "label",
                    "to_name": "text",
                    "type": "labels",
                    "readonly": false,
                    "id": 33,
                    "value": {
                        "start": 866,
                        "end": 879,
                        "text": "the principle",
                        "labels": [
                            "Entity"
                        ],
                        "score": 1
                    }
                },
                {
                    "from_name": "label",
                    "to_name": "text",
                    "type": "labels",
                    "readonly": false,
                    "id": 34,
                    "value": {
                        "start": 896,
                        "end": 918,
                        "text": "a business combination",
                        "labels": [
                            "Entity"
                        ],
                        "score": 1
                    }
                },
                {
                    "from_name": "label",
                    "to_name": "text",
                    "type": "labels",
                    "readonly": false,
                    "id": 35,
                    "value": {
                        "start": 920,
                        "end": 927,
                        "text": "Excerpt",
                        "labels": [
                            "Entity"
                        ],
                        "score": 1
                    }
                },
                {
                    "from_name": "label",
                    "to_name": "text",
                    "type": "labels",
                    "readonly": false,
                    "id": 36,
                    "value": {
                        "start": 933,
                        "end": 973,
                        "text": "ASC Master Glossary\nbusiness combination",
                        "labels": [
                            "Entity"
                        ],
                        "score": 1
                    }
                },
                {
                    "from_name": "label",
                    "to_name": "text",
                    "type": "labels",
                    "readonly": false,
                    "id": 37,
                    "value": {
                        "start": 975,
                        "end": 988,
                        "text": "A transaction",
                        "labels": [
                            "Entity"
                        ],
                        "score": 1
                    }
                },
                {
                    "from_name": "label",
                    "to_name": "text",
                    "type": "labels",
                    "readonly": false,
                    "id": 38,
                    "value": {
                        "start": 992,
                        "end": 1003,
                        "text": "other event",
                        "labels": [
                            "Entity"
                        ],
                        "score": 1
                    }
                },
                {
                    "from_name": "label",
                    "to_name": "text",
                    "type": "labels",
                    "readonly": false,
                    "id": 39,
                    "value": {
                        "start": 1013,
                        "end": 1024,
                        "text": "an acquirer",
                        "labels": [
                            "Entity"
                        ],
                        "score": 1
                    }
                },
                {
                    "from_name": "label",
                    "to_name": "text",
                    "type": "labels",
                    "readonly": false,
                    "id": 40,
                    "value": {
                        "start": 1033,
                        "end": 1040,
                        "text": "control",
                        "labels": [
                            "Entity"
                        ],
                        "score": 1
                    }
                },
                {
                    "from_name": "label",
                    "to_name": "text",
                    "type": "labels",
                    "readonly": false,
                    "id": 41,
                    "value": {
                        "start": 1044,
                        "end": 1066,
                        "text": "one or more businesses",
                        "labels": [
                            "Entity"
                        ],
                        "score": 1
                    }
                },
                {
                    "from_name": "label",
                    "to_name": "text",
                    "type": "labels",
                    "readonly": false,
                    "id": 42,
                    "value": {
                        "start": 1068,
                        "end": 1075,
                        "text": "Excerpt",
                        "labels": [
                            "Entity"
                        ],
                        "score": 1
                    }
                },
                {
                    "from_name": "label",
                    "to_name": "text",
                    "type": "labels",
                    "readonly": false,
                    "id": 43,
                    "value": {
                        "start": 1081,
                        "end": 1084,
                        "text": "ASC",
                        "labels": [
                            "Entity"
                        ],
                        "score": 1
                    }
                },
                {
                    "from_name": "label",
                    "to_name": "text",
                    "type": "labels",
                    "readonly": false,
                    "id": 44,
                    "value": {
                        "start": 1097,
                        "end": 1106,
                        "text": "An entity",
                        "labels": [
                            "Entity"
                        ],
                        "score": 1
                    }
                },
                {
                    "from_name": "label",
                    "to_name": "text",
                    "type": "labels",
                    "readonly": false,
                    "id": 45,
                    "value": {
                        "start": 1131,
                        "end": 1144,
                        "text": "a transaction",
                        "labels": [
                            "Entity"
                        ],
                        "score": 1
                    }
                },
                {
                    "from_name": "label",
                    "to_name": "text",
                    "type": "labels",
                    "readonly": false,
                    "id": 46,
                    "value": {
                        "start": 1148,
                        "end": 1159,
                        "text": "other event",
                        "labels": [
                            "Entity"
                        ],
                        "score": 1
                    }
                },
                {
                    "from_name": "label",
                    "to_name": "text",
                    "type": "labels",
                    "readonly": false,
                    "id": 47,
                    "value": {
                        "start": 1163,
                        "end": 1185,
                        "text": "a business combination",
                        "labels": [
                            "Entity"
                        ],
                        "score": 1
                    }
                },
                {
                    "from_name": "label",
                    "to_name": "text",
                    "type": "labels",
                    "readonly": false,
                    "id": 48,
                    "value": {
                        "start": 1198,
                        "end": 1212,
                        "text": "the definition",
                        "labels": [
                            "Entity"
                        ],
                        "score": 1
                    }
                },
                {
                    "from_name": "label",
                    "to_name": "text",
                    "type": "labels",
                    "readonly": false,
                    "id": 49,
                    "value": {
                        "start": 1216,
                        "end": 1229,
                        "text": "this Subtopic",
                        "labels": [
                            "Entity"
                        ],
                        "score": 1
                    }
                },
                {
                    "from_name": "label",
                    "to_name": "text",
                    "type": "labels",
                    "readonly": false,
                    "id": 50,
                    "value": {
                        "start": 1251,
                        "end": 1261,
                        "text": "the assets",
                        "labels": [
                            "Entity"
                        ],
                        "score": 1
                    }
                },
                {
                    "from_name": "label",
                    "to_name": "text",
                    "type": "labels",
                    "readonly": false,
                    "id": 51,
                    "value": {
                        "start": 1275,
                        "end": 1286,
                        "text": "liabilities",
                        "labels": [
                            "Entity"
                        ],
                        "score": 1
                    }
                },
                {
                    "from_name": "label",
                    "to_name": "text",
                    "type": "labels",
                    "readonly": false,
                    "id": 52,
                    "value": {
                        "start": 1306,
                        "end": 1316,
                        "text": "a business",
                        "labels": [
                            "Entity"
                        ],
                        "score": 1
                    }
                },
                {
                    "from_name": "label",
                    "to_name": "text",
                    "type": "labels",
                    "readonly": false,
                    "id": 53,
                    "value": {
                        "start": 1321,
                        "end": 1331,
                        "text": "the assets",
                        "labels": [
                            "Entity"
                        ],
                        "score": 1
                    }
                },
                {
                    "from_name": "label",
                    "to_name": "text",
                    "type": "labels",
                    "readonly": false,
                    "id": 54,
                    "value": {
                        "start": 1349,
                        "end": 1359,
                        "text": "a business",
                        "labels": [
                            "Entity"
                        ],
                        "score": 1
                    }
                },
                {
                    "from_name": "label",
                    "to_name": "text",
                    "type": "labels",
                    "readonly": false,
                    "id": 55,
                    "value": {
                        "start": 1361,
                        "end": 1381,
                        "text": "the reporting entity",
                        "labels": [
                            "Entity"
                        ],
                        "score": 1
                    }
                },
                {
                    "from_name": "label",
                    "to_name": "text",
                    "type": "labels",
                    "readonly": false,
                    "id": 56,
                    "value": {
                        "start": 1400,
                        "end": 1415,
                        "text": "the transaction",
                        "labels": [
                            "Entity"
                        ],
                        "score": 1
                    }
                },
                {
                    "from_name": "label",
                    "to_name": "text",
                    "type": "labels",
                    "readonly": false,
                    "id": 57,
                    "value": {
                        "start": 1419,
                        "end": 1430,
                        "text": "other event",
                        "labels": [
                            "Entity"
                        ],
                        "score": 1
                    }
                },
                {
                    "from_name": "label",
                    "to_name": "text",
                    "type": "labels",
                    "readonly": false,
                    "id": 58,
                    "value": {
                        "start": 1434,
                        "end": 1454,
                        "text": "an asset acquisition",
                        "labels": [
                            "Entity"
                        ],
                        "score": 1
                    }
                }
            ]
        }
    ]
}
]
  3. Click 'Label All Tasks'
  4. See the error in the browser's console 
    Uncaught (in promise) TypeError: e.id.replace is not a function
at store.js:369:27
at Array.forEach (<anonymous>)
at Object.addAnnotationFromPrediction (store.js:367:9)
at Xn (mobx.module.js:928:19)
at r (mobx.module.js:915:16)
at mobx-state-tree.module.js:2593:35
at Oi (mobx-state-tree.module.js:2456:16)
at Object.r (mobx-state-tree.module.js:2483:16)
at Ih.setAnnotation (lsf-sdk.js:292:25)
at Ih.setLSFTask (lsf-sdk.js:230:10)
at Ih.selectTask (lsf-sdk.js:220:10)
at Ih.loadTask (lsf-sdk.js:205:23)
at async lsf-sdk.js:368:14
at async Promise.all (:8080/projects/2/index 0)

Expected behavior I'd like to be able to import pre-annotations and modify them.

Environment (please complete the following information):

ToddMorrill commented 2 years ago

FWIW, dropping the "id" key from the label dictionaries appears to have resolved the issue.

makseq commented 2 years ago

Ids should be strings.