Question: would basic-pitch be capable of being adapted to work on realtime audio input from a mobile device mic?

spotify / basic-pitch

A lightweight yet powerful audio-to-MIDI converter with pitch bend detection

https://basicpitch.io

Apache License 2.0

3.5k stars 275 forks source link

Question: would basic-pitch be capable of being adapted to work on realtime audio input from a mobile device mic? #108

Closed eedeebee closed 10 months ago

eedeebee commented 10 months ago

Hey I'm curious if the model evaluation would work well enough for real time audio input on iOS and Android - say even inside React Native given the TS implementation. I'd be happy to give this a whirl if this might work. All the examples wire it up to reading audio from files and producint midi = I'm interested in realtime transcription (note and chord) detection from a mobile device mic. Thanks! And if there's a better place to ask questions , please lmk!

eedeebee commented 10 months ago

I see this is a dup of https://github.com/spotify/basic-pitch/issues/19

drubinstein commented 10 months ago

Hi @eedeebee and thanks for your interest. Although this could be used in iOS and Android (I have at the minimum confirmed that the model can be serialized to tflite and coreml ), the model currently cannot be run in real time like how you think of it. You can look at previous issues for reasons why #85 #44 #19