superpoweredSDK / web-audio-javascript-webassembly-SDK-interactive-audio

🌐 Superpowered Web Audio JavaScript and WebAssembly SDK for modern web browsers. Allows developers to implement low-latency interactive audio features into web sites and web apps with a friendly Javascript API. https://superpowered.com
152 stars 16 forks source link

How do you properly cleanup AudioContext and AudioWorkletProcessor? (possible memory leak) #2

Closed svenoaks closed 4 years ago

svenoaks commented 5 years ago

If I try to close the AudioContext and make a new one, along with a new AudioNode, a memory leak happens - it looks like all the raw audio data from the last node is retained in the AudioNode.

See example with slightly modified example_timestretching at https://github.com/svenoaks/superpowered-js-memory-leak.git. Press Start button and let it do it's thing. You'll see the memory from a Google Chrome Helper (or FF equivalent) continually rise.

I can prevent this by never using a new AudioNode, but I think it would be better if I was able to release them properly and create new ones at will.

gaborszanto commented 5 years ago

I checked the web standard documentations and i don't see any "destructor" method specified for this. Superpowered does have destructors but it seems like you need to call them directly after you detached the node.

svenoaks commented 5 years ago

How do I call the destructors?

gaborszanto commented 5 years ago

All the destructors are documented in the documentation: https://superpowered.com/js-wasm-sdk/docs.html For example: some_mixer_object.destruct();

svenoaks commented 5 years ago

I have worked around this with regular AudioContext by retaining the AudioContext and AudioWorkletProcessor when I want to load a new track, and just changing the internal data in the AudioWorkletProcessor. This causes no memory leaks.

However, now I want to also use OfflineAudioContext to render as fast as possible. This works fine but OfflineAudioContext is a one-shot deal, after startRendering() completes there is no way to reuse it. Here I am running into the same issue, huge chunks of memory are being retained when I recreate an OfflineAudioContext and AudioWorkletProcessor each time.

I call all of the following prior to calling disconnect() on the node (or not calling disconnect(), same result), but 100's of MB retained each time

if (typeof message.destruct !== 'undefined') {
            this.destructed = true
            this.clearLoopPoints()
            Superpowered.destroyFloatArray(this.pcm)
            this.pcm = null
            this.timeStretching.destruct()
            this.left = null
            this.right = null
            this.posFrames = null
            this.lengthFrames = null
        }
svenoaks commented 5 years ago

This retention only happens with Chrome with AudioWorklet, FireFox is not having this problem, at least with the OfflineAudioContext.

gaborszanto commented 5 years ago

The linear WASM memory is just 32 MB. If you have 100s of MBs retained then they are something else.

gaborszanto commented 5 years ago

Btw, you can do offline rendering in a simple Worker, no need for OfflineAudioContext.

svenoaks commented 5 years ago

I spent many hours debugging and it seems whatever is passed into AudioWorkletProcessor (even not using Superpowered wrapper) gets retained no matter what you do. The large arrays with the raw PCM (this.left, this.right) never get GC'd even if they are set to null at arbitrary times in the existence of the Node.

I was wondering if it would be be possible to use AudioBufferSourceNode to feed the node with the timestretching, but I think this would not work because there won't be enough input frames delivered to produce output at some rate settings?

Yes, I am going to use a Worklet for the file saving, it will work fine.

gaborszanto commented 5 years ago

Exactly, AudioNodes are operating with fixed buffer sizes defeating time stretching.

gaborszanto commented 4 years ago

I close this because Superpowered does clean up everything. AudioWorkletProcessor implementation problems in web browsers are not Superpowered issues.