Real-Time Recording and Call Detection

DrCoffey / DeepSqueak

DeepSqueak v3: Using Machine Vision to Accelerate Bioacoustics Research

BSD 3-Clause "New" or "Revised" License

373 stars 89 forks source link

Open DrCoffey opened 3 years ago

DrCoffey commented 3 years ago

DeepSqueak currently detects calls in completed audio recordings. However, real-time spectrogram visualization and call detection would be valuable.

[ ] Allow DeepSqueak to record a real-time signal from an external microphone, soundcard, etc.
[ ] Display real-time spectrogram within DeepSqueak App
[ ] Give option to apply detection network to real-time recording
[ ] Display detected calls in the DeepSqueak App
[ ] Develop a method for triggering a hardware output when calls are detected

VoiceScientist commented 3 years ago

Huge fan of this task! Could I add independent multichannel monitoring and hardware triggers to the feature list? Let me know how my lab can help.

DrCoffey commented 3 years ago

Hey @VoiceScientist

vijayiyer05 commented 3 years ago

@DrCoffey, capturing some ideas from our brainstorming today:

Integrating the Audio Toolbox is likely a top choice given its focus on streaming
The implementation may not need (or be easily set up) to use a callback structure as in some other I/O streaming patterns from the past. An emerging pattern is to use parallel processing explicitly, e.g. to ensure apps stay responsive.
For the graphics display, if not using it already, the animated line may be useful to minimize the time for graphics updates while streaming