search

sandrohanea / whisper.net

Whisper.net. Speech to text made simple using Whisper Models
MIT License
534 stars 82 forks source link

Always english #18

Closed PringlesKing closed 1 year ago

PringlesKing commented 1 year ago

When using .WithLanguageDetection() or .WithLanguage("auto") the language is always English auto-detected language: en (p = 0.515557) but if you specify the correct language then everything is fine

Other "port" and "bindings" libraries work fine and detect the language correctly

sandrohanea commented 1 year ago

Hello @PringlesKing, It seems I introduced a bug in the underlying cpp library when using states (so it can be used in parallel).

Thanks for reporting the issue!

Fixed it in both places: https://github.com/sandrohanea/whisper.net/commit/ab2617326eb5e08831ab133d08e8218a28f2a3bf + https://github.com/ggerganov/whisper.cpp/pull/627

Also, added the language on the segment data and a test for this case: https://github.com/sandrohanea/whisper.net/commit/38d1bf3aa72f962bed22067aa8a060bcdd4b24d3

Will wait a bit to see if the cpp PR gets merged soon, otherwise, I'll create a new version of whisper.net with my fork.

Also, will keep you posted when the newer version is available.

PringlesKing commented 1 year ago

There are more bugs here, for example, the temperature is inverted, at 0.9 it behaves like 0.1 for the OpenAI API.

if you specify .WithMaxTokensPerSegment(1) then there will be only one segment also .SplitOnWord() doesn't work at all ¯\_(ツ)_/¯

and absolute silence cannot be detected by any tresholds

sandrohanea commented 1 year ago

The language problem was fixed, and the newer version was published on nuget: https://www.nuget.org/packages/Whisper.net/1.2.2 About the other problems, can you, please, check if you have them also with the cpp library?

If yes, a better report will be directly there, and we can take a look to see what's going on. If the issue is only reproducing on these bindings, please, report a new issue here with the description.