transcript time sync annotation

[keighrim]

1. We will use cadet.
2. Annotation will be done at the sentence or pseudo-sentence (fixed number of tokens or time) level.

[keighrim]

I was looking at the Gabe's annotation data to evaluate gentle forced aligner, and found that text used for cadet annotation is very different from what we were considering as "gold" transcript text. We may need to completely redo the cadet annotation using THE original data as underlying text, unless someone who actually worked on the annotation process (such as @caseyedavis12 or @gmalexander29 ) can guide us to any documentation on what the transformation process was added to the text, so that we can reproduce the same data.

Quick and dirty workaround would be treating the text from cadet annotation as "gold" and re-run the forced aligner, but then we got into a situation where we will have two "golds" under one GUID, and our evaluation framework/code is not currently designed to deal with such situation.

Here's one example of many discrepancies between the original text, cadet text, text from gentle.

• "original" text: https://github.com/clamsproject/aapb-collaboration/blob/3642731e7dd8d1183e16aa70f4c45e93665f2597/21/cpb-aacip-507-4t6f18t178-transcript.txt#L143
• text under annotated data: https://github.com/clamsproject/aapb-annotations/blob/f884e10d0b9d4b1d68e294d83c6e838528d2c249/newshour-transcript-sync/golds/aapb-collaboration-21/cpb-aacip-507-4t6f18t178.tsv#L362-L364
• text from gentle output

  {
        "@type": "http://vocab.lappsgrid.org/Token",
        "properties": {
          "start": 20562,
          "end": 20566,
          "word": "Okay",
          "document": "d2",
          "id": "to_4100"
        }
      },
      ,
      ...
      ,
      {
        "@type": "http://vocab.lappsgrid.org/Token",
        "properties": {
          "start": 20568,
          "end": 20569,
          "word": "I",
          "document": "d2",
          "id": "to_4101"
        }
      },
      ,
      ,
      {
        "@type": "http://vocab.lappsgrid.org/Token",
        "properties": {
          "start": 20570,
          "end": 20575,
          "word": "would",
          "document": "d2",
          "id": "to_4102"
        }
      },
      ,
      ,
      {
        "@type": "http://vocab.lappsgrid.org/Token",
        "properties": {
          "start": 20576,
          "end": 20579,
          "word": "say",
          "document": "d2",
          "id": "to_4103"
        }
      },
      ,
      ,
      {
        "@type": "http://vocab.lappsgrid.org/Token",
        "properties": {
          "start": 20580,
          "end": 20584,
          "word": "that",
          "document": "d2",
          "id": "to_4104"
        }
      },
      ,
      ,
      {
        "@type": "http://vocab.lappsgrid.org/Token",
        "properties": {
          "start": 20585,
          "end": 20590,
          "word": "would",
          "document": "d2",
          "id": "to_4105"
        }
      },
      ,
      ,
      {
        "@type": "http://vocab.lappsgrid.org/Token",
        "properties": {
          "start": 20591,
          "end": 20593,
          "word": "be",
          "document": "d2",
          "id": "to_4106"
        }
      },
      ,
      ,
      {
        "@type": "http://vocab.lappsgrid.org/Token",
        "properties": {
          "word": "uh",
          "document": "d2",
          "id": "to_4107"
        }
      },
      {
        "@type": "http://vocab.lappsgrid.org/Token",
        "properties": {
          "start": 20595,
          "end": 20601,
          "word": "before",
          "document": "d2",
          "id": "to_4108"
        }
      },
      ,
      ,
      {
        "@type": "http://vocab.lappsgrid.org/Token",
        "properties": {
          "start": 20602,
          "end": 20607,
          "word": "check",
          "document": "d2",
          "id": "to_4109"
        }
      },
      ,
      ,
      {
        "@type": "http://vocab.lappsgrid.org/Token",
        "properties": {
          "start": 20608,
          "end": 20611,
          "word": "off",
          "document": "d2",
          "id": "to_4110"
        }
      },
      ,
      ,
      {
        "@type": "http://vocab.lappsgrid.org/Token",
        "properties": {
          "word": "uh",
          "document": "d2",
          "id": "to_4111"
        }
      },
      {
        "@type": "http://vocab.lappsgrid.org/Token",
        "properties": {
          "word": "uh",
          "document": "d2",
          "id": "to_4112"
        }
      },
      {
        "@type": "http://vocab.lappsgrid.org/Token",
        "properties": {
          "start": 20613,
          "end": 20622,
          "word": "somewhere",
          "document": "d2",
          "id": "to_4113"
        }
      },
      ,
      ,
      {
        "@type": "http://vocab.lappsgrid.org/Token",
        "properties": {
          "start": 20623,
          "end": 20629,
          "word": "around",
          "document": "d2",
          "id": "to_4114"
        }
      },
      ,
      ,
      {
        "@type": "http://vocab.lappsgrid.org/Token",
        "properties": {
          "start": 20631,
          "end": 20634,
          "word": "375",
          "document": "d2",
          "id": "to_4115"
        }
      },
      ,
      ,
      {
        "@type": "http://vocab.lappsgrid.org/Token",
        "properties": {
          "start": 20635,
          "end": 20637,
          "word": "or",
          "document": "d2",
          "id": "to_4116"
        }
      },
      ,
      ,
      {
        "@type": "http://vocab.lappsgrid.org/Token",
        "properties": {
          "start": 20639,
          "end": 20642,
          "word": "380",
          "document": "d2",
          "id": "to_4117"
        }
      },

[jarumihooi]

The original issue's name was: timescript time sync annotation guideline. The expected tasks are to add a readme and guideline to this project. This is done with a readme. It has the guideline inside. Therefore its fixed partially by #64 .
However, the issue brought up with Keigh comment about the gold data from annotation and the evaluation gold not matching is still an issue. Therefore, the issue will be renamed and remain open. This issue is linked in the readme.

[keighrim]

One way we can "resolve" this once and for all, is to write a code to align tokens in two transcripts and transfer time stamp annotations over the original gold transcript files. But I expect writing that piece of code will be not so trivial, so not sure it's worthwhile effort.