Real-time pitch adjustment

WebAudio / web-audio-api

The Web Audio API v1.0, developed by the W3C Audio WG

https://webaudio.github.io/web-audio-api/

Other

1.05k stars 168 forks source link

Real-time pitch adjustment #2443

Open svgeesus opened 6 years ago

svgeesus commented 6 years ago

Split out from https://github.com/WebAudio/web-audio-api/issues/938 as a separate issue because this is orthogonal to on-the-fly decoding of compressed loop buffers.

svgeesus commented 5 years ago

Time Stretching And PitchShifting of Audio Signals

svgeesus commented 5 years ago

A Brief History of Pitch Correction in Music

jas-ableton commented 4 years ago

Yes, this would be very useful to have built into web audio sample playback. Being able to vary the playback speed while maintaining a constant pitch, or vary pitch while maintaining a constant playback speed would allow developers to implement lots of very useful workflows that are found in common DAWs (synchronizing recorded audio to a timeline, for example).

In a DAW like Ableton Live, this concept is referred to as "warping": https://www.ableton.com/en/manual/audio-clips-tempo-and-warping/

padenot commented 4 years ago

The variety of possible approaches makes this a good candidate to have (a) prototype(s) using AudioWorklet, to investigate the space in terms of algorithm (PSOLA/WSOLA/phase vocoder, a million others), and API.

This would probably be a new node to have maximal flexibility.

rtoy commented 4 years ago

I think we also agreed that it should be a processing node and not a source node. You feed it some audio, and it will pitch adjust the input.

rtoy commented 4 years ago

TPAC 2020:

@padenot said this has been implemented in an AudioWorklet. Next step is to get the code and see how it works and propose a new node and API for this.

rtoy commented 4 years ago

@padenot found the link: https://github.com/olvb/phaze/

padenot commented 3 years ago

AudioWG virtual F2F summary:

This seems very useful as a basic building block
Two things considered:
- Add preservesPitch defaulting to false to AudioBufferSourceNode -- like its HTMLMediaElement counterpart (but with a default value that is the opposite for compatibility), this changes from resampling to time-stretching
- Add a new node, PitchShiftNode (placeholder name), that changes the pitch of incoming audio. This might need to delay the incoming audio by a number of samples depending on various parameters (algorithm quality, pitch shifting ratio, etc.)