Can you provide an example to merge(or mix) multiple audio stream(buffer) into one track?

[yingshaoxo]

I'm using a client, it uses a package says it Only support PCM 16bit Mono

When I receive it in rust, it becomes Vec<u8>

[253, 255, 47, 0, 34, 0, 39, 0, 49, 0, 64, 0, 35, 0, 60, 0, 52, 0, 70, 0, 41, 0, 31, 0, 53, 0, 15, 0, 25, 0, 10, 0, 27, 0, 7, 0, 13, 0, 10, 0, 6, 0, 22, 0, 251, 255, 14, 0, 23, 0, 16, 0, 11, 0, 20, 0, 30, 0, 17, 0, 18, 0, 34, 0, 20, 0, 28, 0, 11, 0, 7, 0, 7, 0, 32, 0, 3, 0, 19, 0, 249, 255, 15, 0, 255, 255, 23, 0, 13, 0, 21, 0, 15, 0, 11, 0, 13, 0, 6, 0, 18, 0, 5, 0, 10, 0, 249, 255, 235, 255, 246, 255, 219, 255, 224, 255, 221, 255, 203, 255, 215, 291, 255, 186, 255, 192, 255, 186, 255, 205, 255, 187, 255, 207, 255, 182, 255, 214, 255, 185, 255, 227, 255, 187, 255, 212, 255, 202, 255, 215, 255, 197, 221, 255, 197, 255, 218, 255, 206, 255, 211, 255, 239, 255, 228, 255, 221, 255, 211, 255, 204, 255, 201, 255, 206, 255, 211, 255, 203, 250]

use tonic::{transport::Server, Request, Response, Status};

use hello_world::greeter_server::{Greeter, GreeterServer};
use hello_world::{Empty, HelloReply, HelloRequest, VoiceReply, VoiceRequest};

pub mod hello_world {
    tonic::include_proto!("helloworld"); //This is the package name?
}

use futures::stream::StreamExt;
use std::pin::Pin;
use tokio::sync::broadcast;
use tokio_stream::wrappers::BroadcastStream;
use tokio_stream::Stream;

// #[derive(Debug, Default)]
#[derive(Debug)]
pub struct MyGreeter {
    sender: broadcast::Sender<VoiceReply>,
    // receiver: broadcast::Receiver<VoiceReply>,
}

#[tonic::async_trait]
impl Greeter for MyGreeter {
    async fn send_voice(
        &self,
        request: Request<tonic::Streaming<VoiceRequest>>,
    ) -> Result<Response<Empty>, Status> {
        println!("\n\ngot voice: {:?}", request);

        let mut stream = request.into_inner();
        while let Some(data) = stream.next().await {
            // println!("got data: {:?}", data);
            let data = data.expect("error for data");
            self.sender
                .send(VoiceReply { voice: data.voice })
                .expect("sender: it should voice to receiver sent successfully");
        }

        let reply = hello_world::Empty {};
        Ok(Response::new(reply))
    }

    type GetVoiceStream =
        Pin<Box<dyn Stream<Item = Result<VoiceReply, Status>> + Send + Sync + 'static>>;

    async fn get_voice(
        &self,
        request: Request<Empty>,
    ) -> Result<Response<Self::GetVoiceStream>, Status> {
        println!("\n\nrequest voice: {:?}", request);
        // let Empty {} = request.into_inner();

        let rx = self.sender.subscribe();
        let stream = BroadcastStream::new(rx)
            .filter_map(|res| async move { res.ok() })
            .map(Ok);
        let stream: Self::GetVoiceStream = Box::pin(stream);
        let res = Response::new(stream);

        Ok(res)
    }
}

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    let (tx, mut _rx) = broadcast::channel(16);

    let address_string = "0.0.0.0:40051";
    let addr = address_string.parse()?;

    let greeter = MyGreeter {
        sender: tx,
        // receiver: rx,
    };

    println!("Server is running on http://{} ...", address_string);

    Server::builder()
        .add_service(GreeterServer::new(greeter))
        .serve(addr)
        .await?;

    Ok(())
}

I would love to merge all of those incoming audio together in real-time, what should I do?

[AldaronLau]

Thanks for opening this issue!

I just created:

https://github.com/ardaku/fon/blob/4e48fbed2114b8b47b4ed8d62750c29617fb2b6d/examples/mix.rs

Which mixes two audio files in RAW Stereo 32-Bit Float PCM one at 44.1k and one at 48k into a final 48k audio file (same format). It's not as simple as I had thought it would be, so there's some work to be done in the library (thanks for helping me find it!). Most likely I will add the Mixer struct to fon in place of AudioSink.

Let me know if this answers your question, or if you have any follow-up questions.

[AldaronLau]

Since this question has been open for over a week with no activity, I'm closing it. Feel free to re-open if you think it hasn't been answered properly.