search

ebu / benchmarkstt

Open Source AI Benchmarking toolkit for benchmarking speech to text services
MIT License
54 stars 8 forks source link

language support #131

Closed heenaI closed 3 years ago

heenaI commented 4 years ago

Hi, How many languages does the tool support?

EyalLavi commented 4 years ago

The tool is language agnostic. It compares the reference and hypothesis without an understanding of the language used. It was used successfully on Arabic and Russian as well as Latin scripts.

heenaI commented 4 years ago

Hi Eyal, Thank you for your reply. Another question regarding the definition of Refrence and Hypothesis text. I have two two texts Human transcribed and ASR. I want to evaluate ASR based on human transcribed, so I am right in thinking that Human transcribed would be reference and hypothesis is ASR??

heenaI commented 4 years ago

I will test the tool with Hindi and report back the results

EyalLavi commented 4 years ago

Hi Heenal, you are correct: reference = ground truth = human transcript. We'd love your feedback on the tool. Please raise an issue here with any bug/suggestions.