search

meetecho / simple-whep-client

Simple WHEP Client (based on GStreamer's webrtcbin)
GNU General Public License v3.0
6 stars 2 forks source link

Query on testing #2

Closed SanchayanMaity closed 1 year ago

SanchayanMaity commented 1 year ago

Hello,

Apologize if this is the wrong place to ask, and I should be asking this somewhere else.

I am currently working on writing a GStreamer WHEP source element plugin and was using this simple WHEP client and WHEP server to test as a base point first.

To test this, I follow the below procedure

The connection works with WHEP client showing

[WHEP] Negotiation completed
[WHEP] DTLS connecting...
[WHEP] ICE connected
[WHEP] ICE completed
[WHEP] DTLS connected
[WHEP] PeerConnection connected

and WHEP server showing

[ 'Janus OK', 'WHEP REST API OK' ]
  whep:info [abc123] Created new WHEP endpoint +2s
  whep:info [abc123] Subscribing to WHEP endpoint +14s
  janus:info Got an ack to our finalization (tYdAjIwBcH1vg0hM), waiting for result... +16s
  whep:info [tYdAjIwBcH1vg0hM] Completed WHEP negotiation +235ms

However, I never see the pad-added callback whep_incoming_stream getting called. I also have the WHIP server and WHIP client running fine.

Am I missing some fine print in how I should be testing this?

lminiero commented 1 year ago

Are you feeding the Janus Streaming plugin mountpoint associated with the WHEP endpoint with RTP packets? If not, the WHEP setup will create a PeerConnection with Janus successfully, but no packet will be received, since Janus wouldn't have any to send. There's a sample script you can use for testing, which basically just creates a gstreamer pipeline to send RTP to the default (1) mountpoint.

lminiero commented 1 year ago

If you want the WHEP stream to show what WHIP is ingesting, you have to make sure the WHIP publisher is being RTP forwarded to the WHEP mountpoint, as the two use different plugins in Janus. This is described in more detail in this blog post.

SanchayanMaity commented 1 year ago

Thank you very much for the clarification. My bad that I had missed feeding the Janus Streaming plugin mountpoint associated with the WHEP endpoint with RTP packets. Had also missed the blog post.

Have both the simple client and GStreamer WHEP source plugin working.

Closing the issue.

lminiero commented 1 year ago

Happy to help! Curious to see what the official WHEP source plugin will look like: something I never ended up doing in my client was a more configurable pipeline for the incoming media, as I automatically decode and render it instead. Will the plugin allow you to do more with the media instead?

SanchayanMaity commented 1 year ago

The plugin gives out application/x-rtp streams and user can decide the rest of the pipeline. It does not do any decoding at the source itself. Will be upstreaming that after some clean up and review.

There is also an in-progress/talk on an upstream webrtcsrc element. https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/932 which might end up giving both decoded and RTP pads depending on how the discussion goes. It would require a signalling implementation for WHEP but depending on what webrtcsrc decides to do, a future WHEP plugin version might build on that.

SanchayanMaity commented 1 year ago

FWIW merge request is here https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/949

lminiero commented 1 year ago

One important thing to point out is that my WHEP server and client only implement the "server sends the offer" mode, which means the WHEP client expects an offer and sends an answer. In practice, the WHEP specification also allows the client sending an offer instead, which would require a different management of the negotiation process. This distinction will probably need to be taken into account in the whep source element you're working on.

SanchayanMaity commented 1 year ago

One important thing to point out is that my WHEP server and client only implement the "server sends the offer" mode, which means the WHEP client expects an offer and sends an answer. In practice, the WHEP specification also allows the client sending an offer instead, which would require a different management of the negotiation process. This distinction will probably need to be taken into account in the whep source element you're working on.

Yes. Same with the element. Right now, the element does not implement sending an offer. Need to work on that. Will do so, once the first revision gets accepted upstream.