Unable to render Audio Segment component due to no annotations present

[ijemmao]

Background

I'm following this guide for Automatic Speech Recognition using Segments to port into my application. However, whenever I render the component, I get the empty state that says "No more annotations".

If I provide a dummy annotation object in the annotations array, the component will render with no issues. Ideally, I wouldn't want to hardcode an annotation that includes non-related data to my final data labeling.

I'm wondering if I missed a step with the guide that I'm currently using. I've been going through the documentation to help define Tasks, Annotations, etc. Even though I have come across Frontend references, they don't really help explain what's happening in this case.

I'm only using Label Studio Frontend since I have my own backend handling data management.

DataCollectionHome React Component

const DataCollectionHome = (): ReactElement => {
  const config = `
  <View>
  <Labels name="labels" toName="audio">
    <Label value="Speech" />
    <Label value="Noise" />
  </Labels>
  <AudioPlus name="audio" value="$audio"/>
  <TextArea name="transcription" toName="audio"
            rows="2" editable="true"
            perRegion="true" required="true" />
</View>
  `;

  const interfaces = [
    'panel',
    'update',
    'submit',
    'controls',
    'side-column',
    'annotations:menu',
    'annotations:add-new',
    'annotations:delete',
    'predictions:menu',
  ];

  const user = {
    pk: 1,
    firstName: 'James',
    lastName: 'Dean',
  };

  const task = {
    annotations: [],
    predictions: [],
    id: 1,
    data: {
      audio: 'https://igbo-api.s3.us-east-2.amazonaws.com/audio-pronunciations/5f90c35e49f7e863e92b72f2.webm',
    },
  };

  const messages = {
    DONE: 'Done!',
    NO_COMP_LEFT: 'No more annotations',
    NO_NEXT_TASK: 'No more data available for labeling',
    NO_ACCESS: 'You don\'t have access to this task',
  };

  return (
    <>
      <LabelStudioReact
        config={config}
        task={task}
        interfaces={interfaces}
        user={user}
        messages={messages}
      />
      <PersonalStats />
    </>
  );
};

export default DataCollectionHome;

LabelStudioReact React Component

import LabelStudio from '@heartexlabs/label-studio';

const LabelStudioReact = (props: any): ReactElement => {
  const labelStudioContainerRef = useRef();
  const labelStudioRef = useRef();

  useEffect(() => {
    if (labelStudioContainerRef.current) {
      labelStudioRef.current = new LabelStudio(labelStudioContainerRef.current, {
        ...props,
        onSubmitAnnotation: (LS) => {
          console.log('submit', LS)
        },
        onUpdateAnnotation: (LS) => {
          console.log('update', LS)
        },
        onSelectAnnotation: (LS) => {
          console.log('select', LS)
        },
        onEntityCreate: (LS) => {
          console.log('create ', LS)
        },
      });
    }
  }, []);

  return (
    <>
      <div
        id="label-studio"
        ref={labelStudioContainerRef}
      />
    </>
  );
};

export default LabelStudioReact;

Rendered Empty State

Thanks in advance 😊

[bmartel]

Hey @ijemmao 👋 ! Maybe I can help with this. Looking at the code here, can you explain a bit what Component LabelStudioReact is? I just want to make sure what layer that is. Is it something you built or pulled from LabelStudioFrontend library directly?

As an aside, I also have a next app https://github.com/bmartel/ls-next-app I recently put up as a sample reference, which I am going to build out more around the integrations aspect of LabelStudioFrontend, but maybe this can help as a reference point as well. Even as a way to set a baseline understanding between what you are working with in your own application.

[ijemmao]

@bmartel thanks for getting back to me! I just updated my initial issue description to include the LabelStudioReact component which is a custom component that I created that creates a div element that will be used to inject the Label Studio's audio segmentation component.

I can take a look at your project too to serve as a reference 👏🏾

Update: I took a look at your project (great work btw!) and I noticed that you have a task object that has an array of annotation objects hardcoded - could you further explain why you took that approach? My understanding is that the initial state of a task's annotations array should be an empty array because the labeler would then need to start annotating the given task to complete it.

[bmartel]

@ijemmao Thanks for the update and details! With regards to the task object having annotations in the initial state it was done strictly to demonstrate the ability to load an initial state. That's really the only reason, in a more fleshed out scenario you are 100% correct in thinking that you would load an empty state for the annotations, as your annotators will be filling that in as they do their work.

I am going to try a couple things on my end, see if I can replicate your scenario and provide some guidance from there. Once again, thanks so much for reaching out!

[ijemmao]

Hey! I was able to figure out what problem I was facing. Turns out that I wasn't strictly following the typical data structure for annotations, so the system didn't know how to handle it as I was expecting.

I created a PR that shows how I integrated the audio segmentation frontend into my project if others in the future are interested.

Thanks for taking the time to look into this @bmartel

https://github.com/nkowaokwu/igbo-api-admin/pull/297