RustAudio / rodio

Rust audio playback library
Apache License 2.0
1.8k stars 236 forks source link

Support for webassembly builds? #313

Open robert-w-gries opened 4 years ago

robert-w-gries commented 4 years ago

Is there interest in supporting webassembly in rodio? I found the amethyst fork that hasn't been maintained in several months and was interested in getting the wasm branch back into a buildable state.

One issue I found is that the minimp3 project does not currently support wasm due to the slice_deque dependency.

est31 commented 4 years ago

Sure why not? cpal has wasm support too.

glalonde commented 4 years ago

This already works if you remove the mp3 feature: rodio = { git = "https://github.com/RustAudio/rodio" , default-features = false, features = ["vorbis", "flac", "wav"] }

It also works if we use puremp3 as the mp3 decoder. I've tried this, and it seems okay but there are audible artifacts

robert-w-gries commented 4 years ago

I had to modify the rodio Cargo.toml and add the following feature before getting webassembly to work:

[features]
...
wasm-bindgen = ["cpal/wasm-bindgen"]

I tried the following in my project (using raw audio instead of decoders) and was able to get slightly mangled sound:

    let v = raw_data.to_vec();
    let (_stream, stream_handle) = rodio::OutputStream::try_default().unwrap();
    let buffer = SamplesBuffer::new(1,44100, v);
    let sink = Sink::try_new(&stream_handle).unwrap();
    sink.append(buffer);
    sink.sleep_until_end();

However, I see the following error: panicked at 'can't block with web assembly', src/libstd/sys/wasm/condvar.rs:21:9

According to this comment it's possible to use a Promise instead of sleeping. I'll try that and update with my results

robert-w-gries commented 4 years ago

I'm running into the same problem as #310. I am running rodio webassembly on the main thread, so I can't call sleep_until_end(). I can't use a web worker to run the webassembly code because web workers don't have access to the necessary audio apis.

What's strange to me is that cpal works fine without blocking. Is there a way to use rodio without the blocking behavior, or do I need to give up and just use cpal for playback?

glalonde commented 4 years ago

It works if you use a OutputStream: https://github.com/glalonde/wasm_audio/blob/f01b022e5e9a0525f37f556367154aedf19c7dbc/src/lib.rs#L55

Also make sure you compile for release, because it's has choppy playback otherwise. edit: oops didn't see that's what you're already doing. In that case, I have no idea, but it works for me, keeping a reference around. Using sleep_until_end I got the issue you saw, but I don't think you need to call that.

robert-w-gries commented 4 years ago

Thanks @glalonde, the problem was that I was keeping a reference to the OutputStreamHandle instead of the OutputStream. I think rodio has more problems with choppy playback than using cpal directly, but I'm not sure why that is.

I am going to leave this issue open for now because there should be a wasm-bindgen feature added for rodio that enables cpal's wasm-bindgen feature.

robert-w-gries commented 4 years ago

I ran into an issue specific to Chrome where AudioContexts are suspended until there's been user input, such as clicking a play button: https://developers.google.com/web/updates/2017/09/autoplay-policy-changes#webaudio

rodio provides no method for resuming a suspended stream, so I ensured that the OutputStream is created only after receiving user input.

This led me to the thought that rodio should provide methods of suspending and resuming the underlying AudioContext in order to handle this sort of problem and also to conserve system resources when the stream is playing nothing. In my fork, I simply expose the OutputStream::play() and OutputStream::pause() methods that manipulate the underlying AudioContext. I think there should also be a function that creates an OutputStream in an idle state as try_from_device() always immediately calls _stream.play().