search

orottier / web-audio-api-rs

A Rust implementation of the Web Audio API, for use in non-browser contexts
https://docs.rs/web-audio-api/
MIT License
296 stars 16 forks source link

[TAUDIO]Audio Streams under Flutter and React Native #108

Closed Larpoux closed 2 years ago

Larpoux commented 2 years ago

Hi everybody,

This is not an issue. Just an information. We are now working on the port of the Web Audio API under flutter, react native and Microsoft MAUI. We need for that a native implementation of the Web Audio API on several platforms :

We found two existing libraries which can be used :

Banner

Your library seems very interesting. We appreciate that it is dependant on CPAL, and CPAL is dependant of OBOE. This seems a good design.

Unfortunately we are not expert with Rust. We will probably need a little help from you, to start.

Good luck for your library.

Catsvilles commented 2 years ago

@Larpoux Cool idea, will this be open source or closed source? There is already https://github.com/xioxin/lab_sound_flutter for flutter btw

Larpoux commented 2 years ago

Thank you for the link. I am going to look to this flutter plugin. Our license is actually a big problem for us. I insist heavily for a GPL project. But I am very lonely with such a choice. My peers want all MIT

orottier commented 2 years ago

Hi @Larpoux, exciting to hear you are considering to use our library. If you need anything, just let us know. Adding a dual license is something we can discuss. Please realize we are currently working towards a v1.0 release in which our API will be stabilized. After that, we will continue to work on feature completeness within the 1.* branch.

Larpoux commented 2 years ago

This post just to let you know that I am able to run my little example which play 2 seconds of white noise on Flutter iOS, using the web_audio_api_rs. This is really great. Wondeful! Now we must take an important decision :

Do we go with lab-sound

Or with web_audio_api_rs

We understand that web-audio-api-rs is not yet stabilized. We do not really care : what matters is not what this library is today, but what it will be next year. If you have some recommendations on this choice, please tell us.

Note : I cannot run the example on Android because many problems with my Apple Silicon M1 machine : I must use the last Android NDK r23b, but there is something wrong with rust for NDK versions > 21. But I am confident that it will work when Rust will be updated to support the last Android NDK

b-ma commented 2 years ago

Hey @Larpoux ,

This post just to let you know that I am able to run my little example which play 2 seconds of white noise on Flutter iOS, using the web_audio_api_rs.

Cool, great news!

Do you have an idea of the main use-cases for WebAudio in your ecosystem? I can imagine simple sound design / sonification for games but maybe you already have other application domains in mind?

Larpoux commented 2 years ago

In fact I develop Plugin Libraries but I do not know precisely what our developers do with those libraries. An example could be someone who needs to playback directly from the microphone with an equalizer between the mic and the speaker. The latency must be very small for him/her. Someone else wanted to draw the sound volume or the sound frequency during a playback. Someone else wanted to draw a diagram from his/her sound offline (no latency issue, here). Someone else wanted to catch the audio data recorded and send them on the flight on an UDP connexion.

But I am not really sure what they will do with our Flutter Plugin. I just think that many people will find many great utilisations if we offer a way to process audio streams thanks to the WebAudioAPI

orottier commented 2 years ago

Now we must take an important decision

@Larpoux I realize that is a tough call to make. If there's anything we can do to make the choice for our library easier, let us know. We are fully committed to build out and improve on our work. So definitely in a year or so we will have a stable, performant, versatile and user friendly audio library. For your information we are submitting our project for the Web Audio Conference of 2022. We hope to collect some more feedback and collaborators over there

Larpoux commented 2 years ago

Thank you @orottier. Actually, everybody working on the τDio project want to try using your library and prefer not using LabSound. If later we realize that WebAudioApi-rs was not a good choice then it will be possible to change that dependence.

We will certainly need your help. Personally I would love to contribute to WebAudioApi-rs development, but :

But perhaps I will be able to do some very little things. It seems that you are just two developers working hard on this project and I think this is a big responsibility.

orottier commented 2 years ago

That's great to hear.

Based on what you are saying, I believe the best way to contribute would be to try out some of the examples and play around a bit. We are really interested to hear what you run into and what you think we should make more clear/easier to use. You might also run into platform-specific issues, since we have limited availability of hardware for testing.

If you are a bit more comfortable with rust and the audio examples, you could

Larpoux commented 5 months ago

Hi everybody, After a 2 years vacation I am back on Tau. Tau is a flutter plugin that is an interface to web-audio-api.rs

I had a very bad time to setup an environment development for both flutter and rust. But now I have a correct flutter plugin for web-audio-api.rs. Now everything is clean and I am able to run a flutter app under Xcode that uses your lib.

AND I AM EVEN ABLE TO USE THE XCODE DEBUGGER ON AN IOS DEVICE FOR YOUR RUST CODE

Today I am a very happy geek. I will let you know my advancement.

‘Note: @orottier : I am not sure if I must keep this issue open (this is not an issue but an information). And I am not sure that such a thread is just pollution.

orottier commented 5 months ago

Great to hear back from you and happy to hear everything is working out. Please let us know if you run into any issues later, of if you have any other exciting news to share. This issue is already in Closed state so we can keep it here for further reference/discussion.