Pass sdp offer and ICE candidates through Python

[ayertay]

Hello,

Is it possible to pass an offer and ICE candidates to go2rtc? I'd like to use aiortc to send an offer to go2rtc. I have a signaling channel based on MQTT, so I was wondering if I could do the following: Remote Peer's Offer and ICE ->(MQTT)->Raspberry Pi, Raspberry Pi->go2rtc, go2rtc answer and ICE ->Raspberry Pi -> (MQTT) ->Remote Peer.

Thank you!

[AlexxIT]

You can write MQTT to WebSocket proxy for async SDP exchange. Or MQTT to HTTP POST proxy for sync SDP exchange. Check API examples in webrtc.html and webrtc-sync.html.

[ayertay]

Thank you for your response. I'm listening to Websocket and doing the exchange through that.

Here is the websocket address:

websockets.connect(f"ws://{hostname}:{port}/api/ws?src=linux_usbcam&mode=webrtc,mse,mp4,mjpeg")

I send the offer and candidate with the following:

            if mqtt_cand:
                print("candidate arrived")
                for cand_msg in mqtt_cand:
                    await ws.send(json.dumps(cand_msg, separators=(',', ':')))
            if mqtt_offer and (ctr == 0):
                print("Offer Arrived")
                await ws.send(json.dumps(mqtt_offer[0], separators=(',', ':')))
                ctr = 1

with "type": "webrtc/offer" and "type": "webrtc/candidate".

This is the .yaml file:

api:
  listen: ":1984"
  base_path: ""
  static_dir: ""
  origin: ""

ffmpeg:
  bin: "ffmpeg"
  global: "-hide_banner"
  file: "-re -stream_loop -1 -i {input}"
  http: "-fflags nobuffer -flags low_delay -i {input}"
  rtsp: "-fflags nobuffer -flags low_delay -timeout 5000000 -user_agent go2rtc/ffmpeg -rtsp_transport tcp -i {input}"
  output: "-user_agent ffmpeg/go2rtc -rtsp_transport tcp -f rtsp {output}"
  # ... different presets for codecs

hass:
  config: ""

log:
  level: trace
  api: trace
  exec: trace
  ngrok: trace
  rtsp: trace
  streams: trace
  webrtc: trace

ngrok:
  command: ""

rtsp:
  listen: ":8554"
  username: ""
  password: ""

srtp:
  listen: ":8443"

streams:
  linux_usbcam:   ffmpeg:device?video=0#video=h264#raw=-pix_fmt yuv420p

webrtc:
  listen: ":8555"
  candidates: []
  ice_servers:
    - urls: [ "stun:stun.l.google.com:19302" ]
      username: ""
      credential: ""

When peer sends me offer through MQTT, go2rtc receives everything fine, sends answer and candidates, my camera starts, and I see the following:

fps= 30 q=23.0 size=N/A time=00:00:24.33 bitrate=N/A dup=1 drop=0 speed=   1x

but peer says they never get the video, and after 5-6 seconds I get this:

15:13:44.384 TRC [streams] stop reconnect url=ffmpeg:device?video=0#video=h264#raw=-pix_fmt yuv420p
av_interleaved_write_frame(): Broken pipe
frame=  296 fps= 10 q=21.0 Lsize=N/A time=00:00:29.50 bitrate=N/A dup=1 drop=0 speed=1.01x    
video:12250kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown
[libx264 @ 0x55d58a62c4c0] frame I:6     Avg QP:15.45  size:100594
[libx264 @ 0x55d58a62c4c0] frame P:290   Avg QP:17.84  size: 41171
[libx264 @ 0x55d58a62c4c0] mb I  I16..4:  8.9% 42.3% 48.8%
[libx264 @ 0x55d58a62c4c0] mb P  I16..4: 11.4% 37.3%  2.6%  P16..4: 45.3%  0.0%  0.0%  0.0%  0.0%    skip: 3.3%
[libx264 @ 0x55d58a62c4c0] 8x8 transform intra:71.5% inter:37.3%
[libx264 @ 0x55d58a62c4c0] coded y,uvDC,uvAC intra: 92.3% 97.7% 87.1% inter: 52.1% 56.7% 5.0%
[libx264 @ 0x55d58a62c4c0] i16 v,h,dc,p: 13%  8% 58% 21%
[libx264 @ 0x55d58a62c4c0] i8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 14% 10% 34%  9%  6%  6%  5%  7%  9%
[libx264 @ 0x55d58a62c4c0] i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 19% 16% 27%  9%  7%  6%  5%  5%  6%
[libx264 @ 0x55d58a62c4c0] i8c dc,h,v,p: 51% 24% 16%  9%
[libx264 @ 0x55d58a62c4c0] Weighted P-Frames: Y:11.7% UV:0.0%
[libx264 @ 0x55d58a62c4c0] kb/s:3390.08
Conversion failed!
15:13:44.485 TRC [exec] close url="exec:ffmpeg -hide_banner -f v4l2 -i /dev/video0 -pix_fmt yuv420p -c:v libx264 -g 50 -profile:v high -level:v 4.1 -preset:v superfast -tune:v zerolatency -an -user_agent ffmpeg/go2rtc -rtsp_transport tcp -f rtsp rtsp://localhost:8554/42959b42f358e5566114ea407643c532"

Is there a timeout that I need to change? or is this a configuration issue?

Thanks in advance!

[AlexxIT]

go2rtc didn't like something. It stopped the stream.

[AlexxIT]

Please let me know if the problem is relevant