search

ggerganov / whisper.cpp

Port of OpenAI's Whisper model in C/C++
MIT License
35.38k stars 3.61k forks source link

whisper grammar: experimental implementation with boost::spirit #2127

Open fraxy-v opened 5 months ago

fraxy-v commented 5 months ago

An explorative re-implementation of @ejones's parser which IMHO captures the advantages and disadvantages of boost::spirit very well.

ulatekh commented 4 months ago

I'm interested in alternate grammar implementations, since the existing one has odd accuracy issues, especially when it comes to matching whole words from the grammar. But what does this branch do? It doesn't seem to substitute the boost::spirit implementation in place of the existing grammar implementation.

fraxy-v commented 4 months ago

True. It doesn't replace the original, but adds a test_parse method and its implementation in test-grammar-parser.cpp, and a bit of a hand-wavy test test-grammar.cpp which compares the new boost::spirit implementation against the old one. That's the raw stage that I left it in, since I just wanted to see how it would work with boost::spirit.

I guess you can just plug an example input exhibiting these issues into the test code and see if boost::spirit behaves differently and, if it doesn't, if it can be easily fixed. Let me know if you need help with that.

fraxy-v commented 4 months ago

Having said that, I would also say it may be better to fix the problem in the original implementation. I am fairly familiar with the grammar parser, so If you give me an example grammar input demonstrating the oddities, I may be able to suggest a fix.

ulatekh commented 4 months ago

The only grammar I have is a very large one. If I can figure out how to cut it down to where it reproduces the problem, I will. I imagine the problem is that whisper.cpp's grammar is character-based, and if it finds multiple possibilities that begin with the same prefix, but can't decide between them, it terminates recognition at the common prefix, instead of realizing that an incomplete word is not a match.

fraxy-v commented 4 months ago

Okay, but that doesn't sound like a parser bug, unless you have some modification in mind.

ulatekh commented 4 months ago

It's a bug in how whisper.cpp applies grammars, though. It does me no good to get a response back from Whisper that doesn't match the grammar I gave it.

fraxy-v commented 4 months ago

Fair enough, but the application of the grammar is terra incognita for me. Want to help, but I don't know if I could. @ggerganov what do you think?

ggerganov commented 4 months ago

A repro would be needed - without a specific example of the problem, I don't think we can do anything