search

Adirockzz95 / Piwho

Speaker recognition library based on MARF for raspberry pi and other SBCs.
MIT License
54 stars 20 forks source link

Question: Possible to do sound recognition instead of speaker recognition? #1

Closed activityworkshop closed 8 years ago

activityworkshop commented 8 years ago

Hi, nice work with the python wrapper. I understand from the name that you're only interested in recognising who is speaking, not what they've said. But maybe you know how difficult or easy it would be to extend it so that it can match the sound instead?

I mean, if I have wavs of speaker1 saying "whoosh" and "beep" and "neenaw", I would like it to recognise which of those three things speaker1 says into the microphone, but I don't want to recognise speaker1 saying "nawnee" even if it sounds exactly like speaker1, and I don't really care about recognising speaker2 saying anything. Is that something that piwho could do, or would it need MARF to be extended in order to do this?

Adirockzz95 commented 8 years ago

Unfortunately, Piwho/MARF cannot recognize the word/speech. One thing you could do is, train the Speaker1 with wavs of only "Whoosh", Speaker2 with only "beep" and so on. So, for example, if Piwho recognizes Speaker1 then the sound would be "Whoosh".

activityworkshop commented 8 years ago

Thanks for the quick feedback. Yeah, I can try that, but I'm guessing that the features it's using to recognize a speaker will be roughly the same for the same speaker, and not different enough for the different sounds. I'm guessing that it would need different features to work with.

I'll try out what you suggest, and start following the MARF bits which you use. It sounds like you're just taking the MARF code as it is, and not changing it, right? It seems that the MARF code isn't really being updated any more on sourceforge so you may understand it better than anyone!

Adirockzz95 commented 8 years ago

The 'Features' are different methods to process the audio files. This features apply to the entire model and not to the each speaker. You can try various combinations of feature options to get optimal results.

I did change the code a little bit. I exposed the speaker distances to public API (Java) which was not there in original MARF. Piwho function 'get_speaker_scores()' prints this distances on screen.

activityworkshop commented 8 years ago

Ok, thanks a lot. I'll take a closer look :) I also noticed you're using marf 0.3.0.5, rather than the "latest" 0.3.0.6. I assume that's deliberate too.

activityworkshop commented 8 years ago

Sorry to re-open this, but I have another question which I think is very much related. In the recognition phase, have I understood correctly that the recognition works on a pre-recorded test file? That means, there's no possibility for it to work directly on microphone input, in sort of "real time"?

So after the training has finished, the method would be: start recording, speak into the microphone, stop recording, then call piwho to use the saved file, is that right?

What I'd really like to do is have it react more dynamically as I'm speaking into the microphone, so it doesn't know when I'm going to start speaking or stop speaking. But it seems that's not something which MARF can support right now, correct?