Currently, only character sequences containg /[\d\w]/ are considered tokens and therefore can be annotated. To easily support punctuations as tokens and annotated pauses, it's necessary to support a switch to tokenize non-word tokens. This was requested by the DRuKoLa project.
Currently, only character sequences containg
/[\d\w]/are considered tokens and therefore can be annotated. To easily support punctuations as tokens and annotated pauses, it's necessary to support a switch to tokenize non-word tokens. This was requested by the DRuKoLa project.