nyumaya / nyumaya_audio_recognition

Classify audio with neural nets on embedded systems like the Raspberry Pi
https://nyumaya.com
Apache License 2.0
82 stars 14 forks source link

Real-time detection #9

Closed jonsmirl closed 5 years ago

jonsmirl commented 5 years ago

Have you seen how ARM is doing continuous keyword detection in their samples? https://github.com/ARM-software/ML-KWS-for-MCU/blob/master/Deployment/Source/KWS/kws.cpp

This is the core part... they are able to reuse most of the computation from the previous detection and then quickly compute the features on the newly arrived data.

if(num_frames>recording_win) { //move old features left memmove(mfcc_buffer,mfcc_buffer+(recording_winnum_mfcc_features),(num_frames-recording_win)num_mfcc_features); } //compute features only for the newly recorded audio int32_t mfcc_buffer_head = (num_frames-recording_win)num_mfcc_features; for (uint16_t f = 0; f < recording_win; f++) { mfcc->mfcc_compute(audio_buffer+(fframe_shift),&mfcc_buffer[mfcc_buffer_head]); mfcc_buffer_head += num_mfcc_features; }

yodakohl commented 5 years ago

Thanks for the hint but this is already done:

int AudioRecognitionImpl::RunMelDetection(const float* const result,const int mel_len) {

size_t fs = sizeof(float);
float tmp[melcount*melframes];

memcpy(tmp, melwindow+mel_len, (melcount*melframes-mel_len)*fs);
memcpy(tmp + (melcount*melframes-mel_len),result,mel_len*fs);
memcpy(melwindow,tmp,melcount*melframes*fs);

return interpret();

}