DamRsn / NeuralNote

Audio Plugin for Audio to MIDI transcription using deep learning.
Apache License 2.0
1.26k stars 65 forks source link

Proposal for realtime audio #108

Open geraldmwangi opened 1 month ago

geraldmwangi commented 1 month ago

Hi. I am the developer of GuitarMidi-LV2 (https://github.com/geraldmwangi/GuitarMidi-LV2). GuitarMidi does guitar to midi via a bank of elliptic cauer filters instead of CQT. The filterbank is divided into harmonic groups. This allows near realtime tracking of the lower E string notes at buffersize 256 samples at 48khz, using the differences in the overtones which are identical to the fundamentals, as well as the fundamentals themselves

The plugin does monophonic and polyphonic tracking, the later is not accurate and produces false positive midi notes from the overtones of the strings played. In monophonic mode the plugin works quite well, although not 100%.

I am researching neural nets since I would like to use the output of the harmonic filterbank as the input of a neural net to reduce the false positives from the overtones.

Maybe we could join forces and integrate Elliptic Cauer Filters into NeuralNote for realtime transcription?