Closed gogins closed 6 years ago
Seems to show how to load WASM in rendering thread:
The AudioWorklet design is classic upper half/lower half. In Csound's Start method, first the upper half has to be done in the CsoundAudioWorklet, then the lower half has to be done in the CsoundAudioProcessor.
Errors have to be sent asynchronously to an error callback.
Some functions can wait for return values using promises.
This claims to work but I'm having trouble building/running it:
https://github.com/madChopsCoderAu/WASMAudio
I can get it to configure with:
emconfigure ./configure CPPFLAGS="-I/usr/include/eigen3" EIGEN_CFLAGS="-I/usr/include/eigen3"
And the build runs to completion... but the example does not run, looks like there are missing parts.
This https://github.com/madChopsCoderAu/WASMAudio project was updated while I was in Buenos Aires. I am trying it again. These are the steps I took to get it to build and run:
source ~/emsdk/emsdk_env.sh
./configure CPPFLAGS="-I/usr/include/eigen3"
emconfigure ./configure CPPFLAGS="-I/usr/include/eigen3"
make VERBOSE=1
emmake make VERBOSE=1
Edit config.h
to #define HAVE_EMSCRIPTEN
and #define HAVE_EIGEN
.
Install Chrome 66 (first version with AudioWorklet enabled by default).
npm install -g polymer
bower install Polymer/polymer#^2.0.0
polymer install
Copy all in webApp/bower_components
up one level to webApp
. Then
polymer serve
Sheesh, but it works! The WASM code is embedded in the libwasmaudio.js
script.
I'm now working on modularizing csound-extended's WebAssembly build. Now that I understand how this works:
csound_extended(Module).then(function(Module) {
console.log("Module: " + Module + " has been loaded.\n");
});
it builds and runs fine.
To use -s SINGLE_FILE=1
builds and runs, but the csound_extended.js
file is about 12 (!!) times bigger than the WASM files and takes forever to load... not a good way to go.
According to https://github.com/WebAudio/web-audio-api/issues/778, the processor must be defined in a separate file from the node. I'm not sure what is best here as the CMask code is WASM as well as the processor code.
I will have to modularize things but that should solve it.
I have both CsoundAudioWorklet and CsoundAudioProcessor working now. The critical steps were:
em++ -v -O1 -std=c++11 --bind --pre-js ../src/CsoundAudioProcessor_prejs.js --post-js ../src/CsoundAudioProcessor_postjs.js -DINIT_STATIC_MODULES=1 -s WASM=1 -s ASSERTIONS=0 -s "BINARYEN_METHOD='native-wasm'" -s LINKABLE=1 -s RESERVED_FUNCTION_POINTERS=1 -s TOTAL_MEMORY=268435456 -s ALLOW_MEMORY_GROWTH=1 -s BINARYEN_ASYNC_COMPILATION=0 -s NO_EXIT_RUNTIME=0 -s SINGLE_FILE=1 -s EXTRA_EXPORTED_RUNTIME_METHODS='["ccall", "cwrap"]' csound_web_audio.bc libcsound.a ../deps/libsndfile-1.0.25/libsndfile-wasm.a -o CsoundAudioProcessor.js
. In particular, building a single file does the base64 encoding automatically, and turning off asyc compilation is essential.Works! Scrims now plays. On stopping and restarting, I get this:
Uncaught RuntimeError: memory access out of bounds
at wasm-function[3785]:216
at wasm-function[3737]:5
at wasm-function[608]:796
at wasm-function[5888]:13
at invoke_ii (http://localhost:5103/CsoundAudioProcessor.js:10737:32)
at wasm-function[623]:652
at wasm-function[209]:31
at wasm-function[394]:114
at wasm-function[5892]:15
at dynCall_iii_2293 (eval at makeDynCaller (http://localhost:5103/CsoundAudioProcessor.js:8543:19), <anonymous>:4:12)
<WASM UNNAMED> @ wasm-00c041fa-3785:108
<WASM UNNAMED> @ wasm-00c041fa-3737:4
<WASM UNNAMED> @ wasm-00c041fa-608:371
<WASM UNNAMED> @ wasm-00c041fa-5888:8
invoke_ii @ CsoundAudioProcessor.js:10737
<WASM UNNAMED> @ wasm-00c041fa-623:342
<WASM UNNAMED> @ wasm-00c041fa-209:16
<WASM UNNAMED> @ wasm-00c041fa-394:58
<WASM UNNAMED> @ wasm-00c041fa-5892:9
dynCall_iii_2293 @ VM54:4
Csound$PerformKsmps @ VM173:8
process @ CsoundAudioProcessor.js:11797
Seems to be fixed by properly calling csound.Stop and csound.Reset.
TODOs include:
I can now do some performance testing and profiling using Scrims, as I can increase the tempo until I get breakups.
Too many uncontrolled variables with Scrims, adapting thecmask.html
example instead. This test works in NW.js if it uses Chrome 66 or later. The instances build up and the time is displayed.
Couldn't figure out what to do to throw exceptions mandated by the spec, used RangeError instead.
There is a bug where restarting with the audio worklet doesn't work. The problem is CsoundEmbind::Start
hangs the second time. I think the processor is still running.
Anyway it is clear enough that the AudioWorklet implementation runs more smoothly than the ScriptProcessorNode implementation. I need to get it to restart...
Ok, just adding simple FM notes up to see how many instances can run without breaking up:
Trying again with faster note generation, no logging to TextArea, and instance count at dropouts:
I repeated these tests several times with consistent results.
Conclusion: CsoundAudioNode, the AudioWorklet + WebAssembly + Embind implementation of Csound for WebAssembly, is done for now.
The AudioWorklet implementation is not actually quite as performant as the other implementations, but does seem run more steadily and to be more resistant to dropouts caused by events in the browser thread. I wonder if it is possible to speed up the AudioWorkletProcessor.process function by not looking at the input unless necessary. I am testing that now. No help.
These are the only obvious TODOs:
See:
It seems clear that Csound has to run on the audio rendering thread, which has advantages, but must then communicate with the browser or JavaScript context thread by message passing through a port. Otherwise, it seems pretty straightforward.