search

edimuj / cordova-plugin-audioinput

This iOS/Android Cordova/PhoneGap plugin enables audio capture from the device microphone, by in near real-time forwarding audio to the web layer of your application. A typical usage scenario for this plugin would be to use the captured audio as source for a web audio node chain, where it then can be analyzed, manipulated and/or played.
https://github.com/edimuj/app-audioinput-demo
MIT License
161 stars 88 forks source link

[Question] How to detect silence to upload saved file? #84

Open giorgiobeggiora opened 5 years ago

giorgiobeggiora commented 5 years ago

I developed a demo that saves a wav file in the cache directory of the app using the fileUrl option.

The current behaviour is:

the user holds a button > the user talks > the user release the button > upload the file

I want to change this behaviour with

the user taps a button > the user talks > the user stops to talk > upload the file

How is it possible?

In alternative of saving the file with fileUrl, is it possible to obtain a blob or a dataurl of the recording, so i can access the plugin's events?

Thanks.

edimuj commented 5 years ago

I believe that you instead of saving to files, need to stream to Web Audio or directly subscribe to the raw audio events, and then continuously analyze the captured data to determine the average volume for each chunk. For example using the Web audio API: https://stackoverflow.com/questions/21247571/how-to-get-microphone-input-volume-value-with-web-audio-api

You also need to determine and specify what is considered "silent", since there is always some background noise when capturing audio through the microphone, so it may vary a lot depending on the environment of the current user, so it is never zero.

Then you need to package the captured audio in an audio container, for using for example https://www.npmjs.com/package/audiobuffer-to-wav or similar and then you can upload it to the server. Since WAV files are pretty large, you may also want to save it to a file and then convert it to a compressed audio format, and then send the compressed file to the server instead, but that is optional I would say.

So I would say that yes it is possible, but you'll need to do the above to get it to work.

edimuj commented 5 years ago

Since there hasn't been any activity on this for a while now, I'm closing it. Feel free to reopen if need be.

giorgiobeggiora commented 5 years ago

I'm still on this, just i don't have time to work on it now. I will study the problem again in some weeks. Thanks.

Edit: i think that using the saved file instead of using audiobuffer-to-wav is more performant, am I wrong?

edimuj commented 5 years ago

Thanks for the update. If you want the following behavior: "the user stops to talk > upload the file" the audio needs to be analyzed in real-time which means that you most probably cannot use the save to file option of this plugin but have to go for events or Web audio API.

giorgiobeggiora commented 5 years ago

a colleague of mine tried to modify the plugin and he was able to get audioinput events while using the file option. So, why are them not available? Performance?

Edit: we tested on Android devices only.

edimuj commented 5 years ago

I don't think Performance would be an issue simultaneously using events and saving files if correctly implemented.

Maybe your friend would like to share his modifications with the rest of the community via a PR, since this is an open-source project?

giorgiobeggiora commented 5 years ago

here is the zip https://we.tl/t-j7UK5bKS5D

giorgiobeggiora commented 5 years ago

hi @edimuj did you download the zip ? do you have the code now?