Unable to run WebRTC

[gc-robotics]

Hi Brett,

I am now trying the WebRTC bin. Havent made many changes to your config file - it's basically the same:

      stun_server: 'stun://stun.l.google.com:19302'
      element_name: 'sendrecv'  # the name of the webrtc element in the middle of the pipeline, appears in diagnostics
      audio_src_bin_descr: ' audiotestsrc volume=0.3 is-live=true wave=red-noise ! audioconvert ! audioresample ! queue ! opusenc ! rtpopuspay ! application/x-rtp,media=audio,encoding-name=OPUS,payload=96 ! queue '
      video_src_bin_descr: ' videotestsrc is-live=true pattern=ball ! videoconvert ! queue ! vp8enc deadline=1 ! rtpvp8pay ! application/x-rtp,media=video,encoding-name=VP8,payload=97 ! queue '
      video_sink_bin_descr: ' queue ! videoconvert ! ximagesink'
      audio_sink_bin_descr: ' queue ! audioconvert ! alsasink '
      # select the signalling protocol handler, then declare properties for it.

      signalling: webrtc_transport_ws
      #signalling: webrtc_transport_ros

      #parameters for signalling over webrtc_transport_ws:
      signalling_server: 'wss://webrtc.nirbheek.in:8443'
      node_id: 150879
      peer_id: 150879

      #paramters for signalling over webrtc_transport_ros:  # not yet implemented
      sdp_pub_topic: 'webrtc_example_bin/sdp_out'
      sdp_sub_topic: 'webrtc_example_bin/sdp_in'
      ice_pub_topic: 'webrtc_example_bin/ice_in'
      ice_sub_topic: 'webrtc_example_bin/ice_in'

No errors in the logs, even with GST_DEBUG = 4 or 5. I have just this bin running, nothing else that could interfere.

I tried webrtc.nirbheek.in to connect to the peer that should have been created by webrtc_bin but it says "Peer not found".

I am not completely sure if I did the right thing to set peer and node ID to the same value. Not 100% sure what node ID refers to (ros? gst?)

However, I believe the problem is somewhere else. It looks like it is not making an out call at all. Netstat shows that only the webpage/firefox webrtc client (nirbheek) is running: alpha@regulus:~$ netstat -nputw | grep 8443 (Not all processes could be identified, non-owned process info will not be shown, you would have to be root to see it all.) tcp 0 0 192.168.8.100:58476 176.58.110.251:8443 ESTABLISHED 400734/firefox

I see from the source that once a connection is established with STUN, then we should get a callback:

    self.node.get_logger().debug('transport connected callbacks')

but I can see from the log that I never get that message.

The only logs that are webrtc_bin related to init are:

[INFO] [pipeline_node-1]: process started with pid [406284]
[pipeline_node-1] [INFO] [1608193598.880708128] [gst_out]: paths: []
[pipeline_node-1] [INFO] [1608193598.881371733] [gst_out]: plugins: ['rosgstbridge', 'alsa', 'videotestsrc', 'audiotestsrc', 'webrtc', 'nice', 'opus', 'vpx', 'dtls', 'srtp', 'rtp', 'rtpmanager']
[pipeline_node-1] [INFO] [1608193598.881910551] [gst_out]: pkg_list: ['gst_bridge']
[pipeline_node-1] [INFO] [1608193598.882793743] [gst_out]: additional gst plugin paths are []
[pipeline_node-1] [INFO] [1608193598.883323274] [gst_out]: required gst plugins are ['rosgstbridge', 'alsa', 'videotestsrc', 'audiotestsrc', 'webrtc', 'nice', 'opus', 'vpx', 'dtls', 'srtp', 'rtp', 'rtpmanager']
[pipeline_node-1] [INFO] [1608193598.898787353] [gst_out]: plugin "rosgstbridge" added
[pipeline_node-1] [INFO] [1608193598.947244262] [gst_out]: GST Pipeline init completed
[pipeline_node-1] [INFO] [1608193598.948073490] [gst_out]: Pipeline node created
[pipeline_node-1] [INFO] [1608193598.948941324] [gst_out]: Sinmple bins created
[pipeline_node-1] [ERROR] [1608193598.949938938] [gst_out]: building websockets signalling client for webrtc_bin
[pipeline_node-1] [INFO] [1608193598.951421770] [gst_out]: using node_id "150879"
[pipeline_node-1] [INFO] [1608193598.951996671] [gst_out]: using peer_id "150879"
[pipeline_node-1] [INFO] [1608193598.952556966] [gst_out]: using server "wss://webrtc.nirbheek.in:8443"
[pipeline_node-1] [INFO] [1608193598.975335127] [gst_out]: adding section webrtc_bin
[pipeline_node-1] [INFO] [1608193598.976018388] [gst_out]: WebRTC bins created
[pipeline_node-1] [INFO] [1608193598.976573166] [gst_out]: setting pipeline to PLAYING
[pipeline_node-1] [INFO] [1608193598.978937825] [gst_out]: on_status_changed
[pipeline_node-1] 0:00:00.194633271 406284      0x32f0920 WARN         audio-resampler audio-resampler.c:274:convert_taps_gint16_c: can't find exact taps

Any suggestion this time?

Thanks in advance! Cheers, G.

[gc-robotics]

Hi Brett,

Not sure if I am on the right track, but it looks like the code hangs pending here:

self.conn = await websockets.connect(self.server, ssl=sslctx)

(webrtc_transport_ws.py)

Any idea why?

Thanks.

[BrettRD]

The WebRTC example is probably the least polished thing in this repo.

You need to start Nirbheek's browser example first, then feed the ID from the browser into the ROS config, then launch the node.

At that websockets line it's waiting for python to connect to Nirbheek's websocket demo server. Probably check out Nirbheek's websocket demos and have a go without ROS first, their efforts are amazing.

Ideally, I'd like a demo that isn't relying on someone else's demo servers

[gc-robotics]

Hi @BrettRD , thanks for the reply.

Yep, I have understood how the handshake workflow works. I tried it manually on Nirbheek.in.

Unfortunately, as mentioned, my problem is that websockets.connect hangs. No timeout. No failure. No success. Just hangs. And the handshake is not completed.

I am trying to figure out why, but I have no idea for the moment.

If you have any suggestion, please let me know.

[BrettRD]

Thanks for your patience on this one, the bug was in the async behaviour of the pipeline.py While it was busy spin()ning for the ROS node, it locked the websocket out of completing a connection Fixed in commit 7ea98dcebd67dd7438b8f536665406f3982715e7