webrtc-rs / webrtc

A pure Rust implementation of WebRTC
https://webrtc.rs
Apache License 2.0
3.95k stars 347 forks source link

Connection establishes with "ghost" media tracks that I've never added #556

Closed bernardolansing closed 2 months ago

bernardolansing commented 3 months ago

Hello. I am trying to develop a video streamer with this library. For now, it's enough for the streamer to transmit a video track to a single consumer, which is an Android app. The streamer won't receive any tracks from the app. No ICE trickling for now too, pretty basic stuff. What puzzles me is that despite I am still not adding any media tracks to the peer connection, the Android app is receiving two of them.

The implementation of the app should be right, as this streamer was implemented in Python before and it (the app) worked fine. Soon after the connection is established, I get two tracks, one of audio and other of video, both belonging to the same stream:

I/flutter (29064): received stream !!! id = 7e9f18bc-acd8-447f-8902-37f308b1b3bb
I/flutter (29064): video tracks: [Track(id: 4f2c4729-4c5c-49e8-8870-d43f868ed1f9, kind: video, label: video, enabled: true, 
muted: false)]
I/flutter (29064): audio tracks: [Track(id: 5771d359-76ff-425d-a239-b03bec16ea60, kind: audio, label: audio, enabled: true, 
muted: false)]

I'll omit the signalling details as I believe that this part is okay. If you guys find it relevant, I can explain it more deeply.

As of the Rust side, I've made my main() function asynchronous and annotated it with #[tokio::main]. With the offer object, I call the function rtc::start_connection(offer).await. This is the function that handles the connection. Let me show it:

pub async fn start_connection(offer: RTCSessionDescription) -> String {
    let connection = produce_peer_connection().await;

    connection.on_peer_connection_state_change(Box::new(|state| {
        println!("peerConnectionStateChanged {state}");
        Box::pin(async {})
    }));

    connection.on_ice_gathering_state_change(Box::new(|state| {
        println!("iceGatheringStateChanged {state}");
        Box::pin(async {})
    }));

    connection.set_remote_description(offer).await
        .expect("Failed to set received offer as remote description");
    let answer = connection.create_answer(None).await
        .expect("Failed to create answer");
    connection.set_local_description(answer).await
        .expect("Failed to set answer as local description");

    let _ = connection.gathering_complete_promise().await;

    tokio::spawn(check_connection_status(connection.clone()));
    tokio::spawn(check_transceivers(connection.clone()));

    return connection.local_description().await
        .expect("Failed to fetch local description")
        .sdp;
}

async fn produce_peer_connection() -> Arc<RTCPeerConnection> {
    let mut media_engine = MediaEngine::default();
    media_engine.register_default_codecs()
        .expect("Failed to register default codecs.");

    // TODO: check if this registry is really necessary.
    let mut registry = Registry::new();
    registry = register_default_interceptors(registry, &mut media_engine)
        .expect("Failed to register default interceptors");

    let api = APIBuilder::new()
        .with_media_engine(media_engine)
        .with_interceptor_registry(registry)
        .build();

    let ice_servers = RTCIceServer {
        urls: vec![
            String::from("stun:stun.l.google.com:19302"),
            String::from("stun:stun1.l.google.com:19302"),
            String::from("stun:stun2.l.google.com:19302"),
        ],
        ..Default::default()
    };
    let config = RTCConfiguration {
        ice_servers: vec![ice_servers],
        ..Default::default()
    };

    let connection = api.new_peer_connection(config).await
        .expect("Failed to create peer connection");

    Arc::new(connection)
}

async fn check_connection_status(conn: Arc<RTCPeerConnection>) {
    loop {
        sleep(Duration::from_secs(3)).await;
        println!("Connection is alive? {}", conn.connection_state());
    }
}

async fn check_transceivers(conn: Arc<RTCPeerConnection>) {
    sleep(Duration::from_secs(5)).await;
    let transceivers = conn.get_transceivers().await;
    for tr in transceivers {
        println!("transceiver direction: {}, kind: {}", tr.direction(), tr.kind());
    }
}

Well, no media tracks added. check_transceivers will eventually print this:

message received: transceiver direction: sendonly, kind: video
message received: transceiver direction: recvonly, kind: audio
message received: transceiver direction: sendonly, kind: audio

Which is weird, because the app doesn't send anything to the streamer. Perhaps I don't get very well the ideia of transceivers, I thought that they were a "superset" of a media stream? Anyway something's definitely off. The app shows a black screen by the way. check_connection_status will report that the connection is alive (and connected) every so.

I've been making some attempts to actually send a real video track over the connection, but what I get from the Android side is the same black screen. The logs show the same messages: video and audio tracks under a single stream, as if that track was never added. Anyway, I apologize beforehand if that's a silly mistake of mine.