srvk / eesen-transcriber

EESEN based offline transcriber VM using models trained on TEDLIUM and Cantab Research
Apache License 2.0
49 stars 14 forks source link

test2.mp3 #15

Closed fmetze closed 8 years ago

fmetze commented 8 years ago

User can as an example transcribe test2.mp3

We should

1) make sure the documentation is accurate (README.md) - the results don't appear in build/output/test2.* (or do they? only from within the VM, right? what about outside the VM?) 2) provide an example transcription for checking the result 3) make sure this is a reasonable example (the results I got look terrible)

riebling commented 8 years ago

1) Results DO appear there, in total:

2) Sure! Informed by results of fixing 3) 3) Oops. I thought the results were somewhat reasonable. How about a different segment from King Solomon's Mines, one that contains only in-vocabulary words? Because, of course the default system is not going to properly transcribe "infadoos" or "twala". However this fact made it very convenient for testing of the LM "add new words" feature :)

Here's the bad output I'm seeing, and how it should be if 100% perfect:

You're warriors much grow where we have resting on their spears introduce.
By law there was one war just after we destroyed the people that came down upon us but it was a civil war dog a dog.
How was that my lord became my half brother had a brother born at the same birth and have the same woman it is not our custom on hard to suffer twins to live the weak are always must died.
But the mother of looking hit away the people child which was born in the last for her heart and over it and that child is to all the king.

Another configuration (bigger LM, smallest segmentation):

Your warriors must grow wary of resting on their spears introduce.
By law there was one war just after we destroyed the people that came down upon us but it was a civil.
For dog a dog how was that my lord became my half brother had a brother.
You're born at the same birth and of the same woman it is not our custom on hard to suffer twins to live the weaker.
Always must die but the mother of looking hit away the people child which was born in the last for her heart and over it and.
The child is to oil leaking.

Yet another config (bigger LM, default segmentation)

Your warriors much grow wary of resting on their spears introduce.
By law there was one war just after we destroyed the people that came down upon us but it was a civil war dog a dog.
How was that my lord became my half brother had a brother born at the same birth and of the same woman it is not our custom on hard to suffer twins to live the weaker always must die.
But the mother of looking hit away the people child which was born in the last for her heart and over it and that child is to oil leaking.

Gold std. output:

Your warriors must grow weary of resting on their spears, Infadoos.
My lord, there was one war, just after we destroyed the people that came down upon us, but it was a civil war; dog ate dog.
How was that?
My lord the king, my half-brother, had a brother born at the same birth, and of the same woman.
It is not our custom, my lord, to suffer twins to live; the weaker always must die.
But the mother of the king hid away the feeble child, which was born the last, for her heart yearned over it, and that child is Twala the king.
fmetze commented 8 years ago

Ok, thanks - this is weird stuff even when one has the reference. Just thought it would be useful to have the reference somewhere so that the user can verify that his system works. No need to add “Infadoos”, but if the reference has been added and documented, that is good.

On Aug 4, 2016, at 3:03 PM, riebling notifications@github.com wrote:

1) Results DO appear there, in total:

build/output/test2.* from the host computer 'vagrant up' home folder ~/tools/eesen-offline-transcriber/build/output/ (for short, and conveninece, and on vagrant's search path) ~/bin/build/output 2) Sure! Informed by results of fixing 3) 3) Oops. I thought the results were somewhat reasonable. How about a different segment from King Solomon's Mines, one that contains only in-vocabulary words? Because, of course the default system is not going to properly transcribe "infadoos" or "twala". However this fact made it very convenient for testing of the LM "add new words" feature :)

Here's the bad output I'm seeing, and how it should be if 100% perfect:

You're warriors much grow where we have resting on their spears introduce. By law there was one war just after we destroyed the people that came down upon us but it was a civil war dog a dog. How was that my lord became my half brother had a brother born at the same birth and have the same woman it is not our custom on hard to suffer twins to live the weak are always must died. But the mother of looking hit away the people child which was born in the last for her heart and over it and that child is to all the king. Another configuration (bigger LM, smallest segmentation):

Your warriors must grow wary of resting on their spears introduce. By law there was one war just after we destroyed the people that came down upon us but it was a civil. For dog a dog how was that my lord became my half brother had a brother. You're born at the same birth and of the same woman it is not our custom on hard to suffer twins to live the weaker. Always must die but the mother of looking hit away the people child which was born in the last for her heart and over it and. The child is to oil leaking. Yet another config (bigger LM, default segmentation)

Your warriors much grow wary of resting on their spears introduce. By law there was one war just after we destroyed the people that came down upon us but it was a civil war dog a dog. How was that my lord became my half brother had a brother born at the same birth and of the same woman it is not our custom on hard to suffer twins to live the weaker always must die. But the mother of looking hit away the people child which was born in the last for her heart and over it and that child is to oil leaking. Gold std. output:

Your warriors must grow weary of resting on their spears, Infadoos. My lord, there was one war, just after we destroyed the people that came down upon us, but it was a civil war; dog ate dog. How was that? My lord the king, my half-brother, had a brother born at the same birth, and of the same woman. It is not our custom, my lord, to suffer twins to live; the weaker always must die. But the mother of the king hid away the feeble child, which was born the last, for her heart yearned over it, and that child is Twala the king. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/srvk/eesen-transcriber/issues/15#issuecomment-237651461, or mute the thread https://github.com/notifications/unsubscribe-auth/AEnA8YYK-ztwVdgHbno6PuZZdFntAiwUks5qcjefgaJpZM4JcQdw.

Florian Metze http://www.cs.cmu.edu/directory/florian-metze Associate Research Professor Carnegie Mellon University

riebling commented 8 years ago

How about this instead: System output

Things will change in ways that they're fragile environment simply can't support and that leads to starvation it leads to uncertainty at leads unrest so the climate change is will be terrible for them.

gold std:

Things will change in ways that their fragile environment simply can't support.
And that leads to starvation it leads to uncertainty it leads to unrest.
So that climate changes will be terrible for them.
riebling commented 8 years ago

Changed to use Bill Gates lecture audio Updated README.md to show expected output (link to repository wiki page) and results