search

spotify / basic-pitch-ts

A lightweight yet powerful audio-to-MIDI converter with pitch bend detection.
https://basicpitch.io
Apache License 2.0
212 stars 15 forks source link

Example code for integrate with React #6

Open neviaumi opened 1 year ago

neviaumi commented 1 year ago

As a caption

I think it would better to have some show case / references implementation about how to use this module .

for example, if would be great if the official one can be open sourced

fheyen commented 1 year ago

I agree that an example for how to use it in a browser would be very helpful, especially one that works real-time!

I wonder if handling incoming audio in separate chunks would work fine or if there has to be some overlap?

drubinstein commented 1 year ago

Hi @davidNHK and @fheyen. We can't currently open source basicpitch.io as it contains some Spotify-internal React components that are not open sourced yet. The website largely uses a similar structure to what exists in the README's usage section where a BasicPitch object is instantiated, data is prepared and then the data is evaluated. Once you get the midi output (the last bit of the usage example in the README), you can take that output and write it to a midi file or put it into your web-based synth of choice.

@fheyen unfortunately, even in the website, Basic Pitch is designed with a final output step that uses the inference run of over the entire file. We believe it may be possible to sacrifice quality and run that step on chunks at a time, but haven't had time to prioritize testing the approach. Thankfully, depending on machine, Basic Pitch is pretty fast (at least it can process a minute of audio in 1/3 of a minute on most machines). There's been similar issues posted over in the Python Basic Pitch repo so you're not alone in the idea.

If either of you want to contribute better examples or even take a stab at chunk-based processing we'll appreciate it :)

fheyen commented 1 year ago

Thanks @drubinstein for the quick reply, in the meantime I was able to get basic-pitch to run in ObservableHQ: https://observablehq.com/@fheyen/hello-spotify-basic-pitch-ts Feel free to link to this notebook in your readme.

mike3osei commented 1 year ago

thanks @fheyen really found this useful in applying this typescript implementation!

westonsoftware commented 5 months ago

Hi, has anyone followed up with this cool idea? Short chunks converted in real-time, even if the delay was pretty long would be fun as a voice to MIDI controller. So not using a file but a mic with a WebAudio buffer.

ttys026 commented 3 months ago

I wrote an online demo shows how to get json and midi format data from basic pitch https://codesandbox.io/p/sandbox/modest-benji-k2nfg5?file=%2Fsrc%2Findex.ts

westonsoftware commented 3 months ago

I wrote an online demo shows how to get json and midi format data from basic pitch https://codesandbox.io/p/sandbox/modest-benji-k2nfg5?file=%2Fsrc%2Findex.ts

@ttys026 Thanks for this ... your midi-canvas looks nice too.