search

airenas / list

Lithuanian Speech Transcription
BSD 3-Clause "New" or "Revised" License
1 stars 1 forks source link

fix.lattice.time: overlaped segments results in incorrect insertion of silence segment #1

Open lv-gh opened 2 months ago

lv-gh commented 2 months ago

lattice.txt:

# 1 S0001
1 1.0 2.0 vienas
1 2.0 3.0 du
1 3.0 4.0 trys
1 4.0 5.0 keturi
1 5.0 6.0 penki
1 6.0 7.0 šeši
1 7.0 8.0 septyni
1 8.0 9.0 aštuoni
1 9.0 10.0 devyni
1 10.0 11.0 dešimt

# 2 S0002
1 2.0 3.0 dar vienas
1 3.0 4.0 dar du
1 4.0 5.0 dar trys
1 5.0 6.0 dar keturi

# 3 S0003
1 20.0 21.0 vienas
1 21.0 22.0 vienas
1 22.0 23.0 karvės
1 23.0 24.0 pienas

$ fix.lattice.time -l 25 lattice.txt > lattice.fix.txt

... Open file lattice.txt ... Fixing ... Insert segment from 0.00, to 1.00 ... Insert segment from 6.00, to 20.00 ... Insert segment from 24.00, to 25.00 ... Done

lattice.fix.txt:

# 1 TYLA
1 0.00 1.00 <tyla>

# 2 S0001
1 1.0 2.0 vienas
1 2.0 3.0 du
1 3.0 4.0 trys
1 4.0 5.0 keturi
1 5.0 6.0 penki
1 6.0 7.0 šeši
1 7.0 8.0 septyni
1 8.0 9.0 aštuoni
1 9.0 10.0 devyni
1 10.0 11.0 dešimt

# 3 S0002
1 2.0 3.0 dar vienas
1 3.0 4.0 dar du
1 4.0 5.0 dar trys
1 5.0 6.0 dar keturi

# 4 TYLA
1 6.00 20.00 <tyla>

# 5 S0003
1 20.0 21.0 vienas
1 21.0 22.0 vienas
1 22.0 23.0 karvės
1 23.0 24.0 pienas

# 6 TYLA
1 24.00 25.00 <tyla>
airenas commented 1 month ago

@lv-gh - thank you.

The lattice format is our internal format, and it is not documented, sorry about that.

The format of the word line is: is it the main flow, from, to, word. The lattice with overlapping time in the main flow is not valid.

Is it a bug with fix.lattice.time? - NO Could fix.lattice.time work more correctly and return failure in your case? - YES

lv-gh commented 1 month ago

No need for failure indeed, but overlapping (overlapped speech) is pretty valid/common using pyannote diarization pipelines (and possibly others), so ideally it should just deal with that, inserting silence blocks/intervals from the farthest word read so far, not from the last (overlapped) block word.

lv-gh commented 1 month ago

Things to add to enhancement; now also fix.segments are incompatible with overlapped speech (segments gets truncated and under certain conditions badly), i.e. currently it's incompatible with pyannote diarization (at least it should be documented, IMHO), which produces overlapping segments.