search

bootphon / pygamma-agreement

Gamma Agreement in Python
MIT License
40 stars 8 forks source link

Corpus shuffling tool #17

Closed Rachine closed 2 years ago

Rachine commented 3 years ago

It would be very beneficial, for potential extension of the gamma, to have the corpus shuffling tool introduced in this paper: Manual Corpus Annotation: Giving Meaning to the Evaluation Metrics link There used to be available code link It allows to test for different type of errors:

  1. False negative
  2. False positive
  3. Fragmentation (Which I think the Pygamma is not taking into account and would be very beneficial to account for)
  4. Shift
  5. Category mistake