Error when viewing TimeSeries label task

[PaulHSch]

Describe the bug In the newest development Environment, when opening a TimeSeries task for labeling the tab freezes/gets stuck on loading. The console gets flooded with a mobx-state-tree error stating that It's trying to read or write to an object that is no longer part of a state tree. (See Screenshot).

This happens specifically when clicking on a single task, when "Label all tasks" is used the TimeSeries still doesn't load, and the console instead shows: "[mobx.array] Attempt to read an array index (0) that is out of bounds (0). Please check length first. Out of bound indices will not be tracked by MobX instrument.js:109:45".

To Reproduce Steps to reproduce the behavior:

1. Create Project with csv data and a TimeSeries template (e.g. Activity Recognition)
2. In the project click on the task
3. Now it usually gets stuck loading

Expected behavior TimeSeries should render after a short time

Screenshots

Environment (please complete the following information):

• OS: Linux / Lubuntu
• Label Studio Version: newest Version from Github / 1.5.0dev (The one you get when you follow the steps for Install for local development)

[makseq]

@PaulSch01 Is it possible to share labeling config and csv sample data where you have this problem?

[PaulHSch]

Yes, of course. It doesn't always freeze using this example data and config, but I'd say 80% of the time it does for me. data: data.csv config:

<View>
    <!-- Control tag for region labels -->
    <TimeSeriesLabels name="label" toName="ts">
        <Label value="Run" background="red"/>
        <Label value="Walk" background="green"/>
        <Label value="Fly" background="blue"/>
        <Label value="Swim" background="#f6a"/>
        <Label value="Ride" background="#351"/>
    </TimeSeriesLabels>

    <!-- Object tag for time series data source -->
    <TimeSeries name="ts" valueType="url" value="$timeseriesUrl" timeColumn="Time">

        <Channel column="data" units="lbs" strokeColor="#1f77b4" legend="Value"/>
    </TimeSeries>
</View>

[bmartel]

Hey @PaulSch01 👋 . I'll take a look into it, this appears to be an issue with the quickview initializing the data graph twice, and in some circumstances its causing a runaway hydration.

[harrymander]

Just adding that I am seeing the same behaviour if it helps: the You are trying to read or write to an object that is no longer part of a state tree seems to be printed for the same number of samples in the time series; i.e., I opened a time series with 9311 samples and got the following warning in the console 9311 times:

instrument.js:109 Error: [mobx-state-tree] You are trying to read or write to an object that is no longer part of a state tree. (Object type: 'TimeSeriesModel', Path upon death: '/annotationStore/root/children/7', Subpath: 'timeformat', Action: ''). Either detach nodes first, or don't use objects after removing / replacing them in the tree.
    at ya (mobx-state-tree.module.js:3427:18)
    at t.assertAlive (mobx-state-tree.module.js:1510:21)
    at t.unbox (mobx-state-tree.module.js:1580:14)
    at t.dehanceValue (mobx.module.js:1040:25)
    at t.get (mobx.module.js:1092:21)
    at e.read (mobx.module.js:4136:37)
    at Object.get [as timeformat] (mobx.module.js:4377:36)
    at get dataObj (TimeSeries.js:191:26)
    at e.computeValue (mobx.module.js:1280:43)
    at e.get (mobx.module.js:1210:35)
    at e.read (mobx.module.js:4136:37)
    at Object.get (mobx.module.js:4400:68)
    at Object.updateValue (TimeSeries.js:512:21)
      (anonymous) @ instrument.js:109
      ya @ mobx-state-tree.module.js:3427
      t.assertAlive @ mobx-state-tree.module.js:1510
      t.unbox @ mobx-state-tree.module.js:1580
      t.dehanceValue @ mobx.module.js:1040
      t.get @ mobx.module.js:1092
      e.read @ mobx.module.js:4136
      get @ mobx.module.js:4377
      get dataObj @ TimeSeries.js:191
      e.computeValue @ mobx.module.js:1280
      e.get @ mobx.module.js:1210
      e.read @ mobx.module.js:4136
      get @ mobx.module.js:4400
      updateValue @ TimeSeries.js:512
      await in updateValue (async)
      Xn @ mobx.module.js:928
      r @ mobx.module.js:915
      (anonymous) @ mobx-state-tree.module.js:2593
      Ti @ mobx-state-tree.module.js:2456
      r @ mobx-state-tree.module.js:2483
      (anonymous) @ store.js:224
      n @ Tree.tsx:294
      n @ Tree.tsx:301
      Jc @ Tree.tsx:308
      i @ store.js:206
      Xn @ mobx.module.js:928
      r @ mobx.module.js:915
      (anonymous) @ mobx-state-tree.module.js:2593
      Ti @ mobx-state-tree.module.js:2456
      r @ mobx-state-tree.module.js:2483
      initializeStore @ AppStore.js:601
      Xn @ mobx.module.js:928
      r @ mobx.module.js:915
      (anonymous) @ mobx-state-tree.module.js:2593
      Ti @ mobx-state-tree.module.js:2456
      r @ mobx-state-tree.module.js:2483
      dge @ configureStore.js:35
      await in dge (async)
      createApp @ LabelStudio.js:58
      mge @ LabelStudio.js:40
      initLabelStudio @ lsf-sdk.js:173
      await in initLabelStudio (async)
      _h @ lsf-sdk.js:164
      initLSF @ dm-sdk.js:379
      (anonymous) @ Label.js:71
      (anonymous) @ Label.js:85
      Gl @ react-dom.production.min.js:262
      t.unstable_runWithPriority @ scheduler.production.min.js:18
      Ho @ react-dom.production.min.js:122
      jl @ react-dom.production.min.js:261
      hl @ react-dom.production.min.js:243
      (anonymous) @ react-dom.production.min.js:123
      t.unstable_runWithPriority @ scheduler.production.min.js:18
      Ho @ react-dom.production.min.js:122
      zo @ react-dom.production.min.js:123
      qo @ react-dom.production.min.js:122
      wl @ react-dom.production.min.js:244
      Ve @ mobx.module.js:1919
      Ze @ mobx.module.js:1895
      He @ mobx.module.js:1595
      (anonymous) @ mobx.module.js:981
      Qe @ mobx.module.js:935
      r @ mobx.module.js:915
      (anonymous) @ mobx-state-tree.module.js:2593
      uo @ mobx-state-tree.module.js:2456
      l @ mobx-state-tree.module.js:3617
      u @ mobx-state-tree.module.js:3631
      Promise.then (async)
      A @ mobx-state-tree.module.js:3671
      u @ mobx-state-tree.module.js:3640
      a @ mobx-state-tree.module.js:3623
      Qe @ mobx.module.js:928
      r @ mobx.module.js:915
      (anonymous) @ mobx-state-tree.module.js:2593
      uo @ mobx-state-tree.module.js:2456
      (anonymous) @ mobx-state-tree.module.js:3626
      n @ mobx-state-tree.module.js:3619
      Qe @ mobx.module.js:928
      r @ mobx.module.js:915
      (anonymous) @ mobx-state-tree.module.js:2593
      uo @ mobx-state-tree.module.js:2456
      r @ mobx-state-tree.module.js:2483
      qh @ app-create.js:73
      await in qh (async)
      initApp @ dm-sdk.js:372
      ew @ dm-sdk.js:183
      (anonymous) @ DataManager.js:51
      (anonymous) @ DataManager.js:84
      await in (anonymous) (async)
      (anonymous) @ DataManager.js:149
      Dl @ react-dom.production.min.js:262
      t.unstable_runWithPriority @ scheduler.production.min.js:18
      Bi @ react-dom.production.min.js:122
      Ll @ react-dom.production.min.js:261
      (anonymous) @ react-dom.production.min.js:261
      R @ scheduler.production.min.js:16
      w.port1.onmessage @ scheduler.production.min.js:12

I can load a file with 572664 samples within a second or two via the Label all tasks interface and get only the following logs:

instrument.js:109 [mobx.array] Attempt to read an array index (0) that is out of bounds (0). Please check length first. Out of bound indices will not be tracked by MobX
    (anonymous) @ instrument.js:109
    get @ mobx.module.js:3369
    get @ mobx.module.js:3053
    setAnnotation @ lsf-sdk.js:345
    setLSFTask @ lsf-sdk.js:294
    selectTask @ lsf-sdk.js:277
    a @ lsf-sdk.js:247
    await in a (async)
    loadTask @ lsf-sdk.js:262
    (anonymous) @ lsf-sdk.js:440
    await in (anonymous) (async)
    (anonymous) @ events.ts:33
    invoke @ events.ts:33
    afterCreate @ AppStore.js:292
    Xn @ mobx.module.js:928
    r @ mobx.module.js:915
    (anonymous) @ mobx-state-tree.module.js:2593
    Ti @ mobx-state-tree.module.js:2456
    r @ mobx-state-tree.module.js:2483
    (anonymous) @ mobx-state-tree.module.js:1458
    Jn @ mobx.module.js:1012
    t.fireHook @ mobx-state-tree.module.js:1457
    t.createObservableInstance @ mobx-state-tree.module.js:1378
    Xn @ mobx.module.js:928
    r @ mobx.module.js:915
    t.createObservableInstanceIfNeeded @ mobx-state-tree.module.js:1316
    t.getValue @ mobx-state-tree.module.js:1912
    get @ mobx-state-tree.module.js:1016
    e.create @ mobx-state-tree.module.js:1836
    Xn @ mobx.module.js:928
    r @ mobx.module.js:915
    t.create @ mobx-state-tree.module.js:1909
    Xn @ mobx.module.js:928
    r @ mobx.module.js:915
    dge @ configureStore.js:30
    await in dge (async)
    createApp @ LabelStudio.js:58
    mge @ LabelStudio.js:40
    initLabelStudio @ lsf-sdk.js:173
    await in initLabelStudio (async)
    _h @ lsf-sdk.js:164
    initLSF @ dm-sdk.js:379
    (anonymous) @ Label.js:71
    (anonymous) @ Label.js:85
    Gl @ react-dom.production.min.js:262
    t.unstable_runWithPriority @ scheduler.production.min.js:18
    Ho @ react-dom.production.min.js:122
    jl @ react-dom.production.min.js:261
    Ge @ react-dom.production.min.js:292
    Zt @ react-dom.production.min.js:73
    r @ helpers.js:72

instrument.js:109 deserializeAnnotation() is deprecated. Use deserializeResults() instead
    (anonymous) @ instrument.js:109
    deserializeAnnotation @ Annotation.js:966
    Xn @ mobx.module.js:928
    r @ mobx.module.js:915
    (anonymous) @ mobx-state-tree.module.js:2593
    Ti @ mobx-state-tree.module.js:2456
    r @ mobx-state-tree.module.js:2483
    addAnnotationFromPrediction @ store.js:401
    Xn @ mobx.module.js:928
    r @ mobx.module.js:915
    (anonymous) @ mobx-state-tree.module.js:2593
    Ti @ mobx-state-tree.module.js:2456
    r @ mobx-state-tree.module.js:2483
    setAnnotation @ lsf-sdk.js:359
    setLSFTask @ lsf-sdk.js:294
    selectTask @ lsf-sdk.js:277
    a @ lsf-sdk.js:247
    await in a (async)
    loadTask @ lsf-sdk.js:262
    (anonymous) @ lsf-sdk.js:440
    await in (anonymous) (async)
    (anonymous) @ events.ts:33
    invoke @ events.ts:33
    afterCreate @ AppStore.js:292
    Xn @ mobx.module.js:928
    r @ mobx.module.js:915
    (anonymous) @ mobx-state-tree.module.js:2593
    Ti @ mobx-state-tree.module.js:2456
    r @ mobx-state-tree.module.js:2483
    (anonymous) @ mobx-state-tree.module.js:1458
    Jn @ mobx.module.js:1012
    t.fireHook @ mobx-state-tree.module.js:1457
    t.createObservableInstance @ mobx-state-tree.module.js:1378
    Xn @ mobx.module.js:928
    r @ mobx.module.js:915
    t.createObservableInstanceIfNeeded @ mobx-state-tree.module.js:1316
    t.getValue @ mobx-state-tree.module.js:1912
    get @ mobx-state-tree.module.js:1016
    e.create @ mobx-state-tree.module.js:1836
    Xn @ mobx.module.js:928
    r @ mobx.module.js:915
    t.create @ mobx-state-tree.module.js:1909
    Xn @ mobx.module.js:928
    r @ mobx.module.js:915
    dge @ configureStore.js:30
    await in dge (async)
    createApp @ LabelStudio.js:58
    mge @ LabelStudio.js:40
    initLabelStudio @ lsf-sdk.js:173
    await in initLabelStudio (async)
    _h @ lsf-sdk.js:164
    initLSF @ dm-sdk.js:379
    (anonymous) @ Label.js:71
    (anonymous) @ Label.js:85
    Gl @ react-dom.production.min.js:262
    t.unstable_runWithPriority @ scheduler.production.min.js:18
    Ho @ react-dom.production.min.js:122
    jl @ react-dom.production.min.js:261
    Ge @ react-dom.production.min.js:292
    Zt @ react-dom.production.min.js:73
    r @ helpers.js:72```

[anishnag]

I have this issue too which appeared only after I deleted annotations.

[bmartel]

I generated an internal ticket to get this looked into further. The issue across all reported cases comes down to the initialization methods used for the data between label stream (Label all tasks button) and quick view (selecting a task in the data manager table). As of right this moment I can't pinpoint the exact issue, but based on the logs seen here the data in the fast loading case (label stream) is doing so because the data is loaded directly into memory, where as the slow case it is getting caught in what looks like a data merge that is firing a handful of other calculations, when it should reasonably just be doing a strict replacement.

[nicky508]

Is there any new on this issue or a workaround? So label studio could be used for time serie annotation?

[nicky508]

Ok, I figured out by removing the timezone information after the datetime seems to solve the issue.

[RASCampbell]

Hello, I think I am having a similar issue. I have recently started using Label Studio (v1.11.0) and I am trying to label a collection of single time-series data (i.e. one time series per task). The data is in a csv file with the format as below:

The data imports fine into Label Studio's Data Manager, however when I try to begin the labelling task for any of the time series (either by choosing a single task or 'Label All Tasks') the labelling screen is stuck on a 'loading' screen (see below)

Looking at the console, I get a similar error to the original poster (see below)

Of note, I have tried Projects for image or text annotation and they work fine. I have also replicated the above error when I have changed the csv(s) to tsv(s).

I'd be grateful if anyone has any advice. Thanks!

[falknerdominik]

Any update on this?

[pmayostendorp]

Ok, I figured out by removing the timezone information after the datetime seems to solve the issue.

Set up a new issue for this specific datetime format issue, which could be masking some of this behavior. https://github.com/HumanSignal/label-studio/issues/6589