Issue with RTSP streaming of x265 or hevc_nvenc encoded file with opus audio: Only audio for WebRTC played and only the first picture for MSE shown.

Dear distinguish team,

I am writing to seek your assistance with a technical issue I am experiencing while testing the stream of an H.265 + opus file on Safari. I have already allowed WebRTC h265 in "Experimental Features" and tried H.264 streaming, which works well using the same settings in the go2rtc.yaml and ffmpeg commands for streaming. To test the h.265 file, I used a file from one of the Apple Xcode open-source projects I work with, but the result was no audio, no video played for WebRTC, and only the first frame opened for MSE. I have included the details of the test below:

rtsp stream: % ./ffmpeg -re -stream_loop -1 -i puppets_with_greenbg_hevc_nvenc_opus.mp4 -c copy -f rtsp rtsp://localhost:8554/stream1

go2rtc.yaml:

streams:
  stream1: ""

info: http://127.0.0.1:1984/api/streams?src=stream1:

{
  "producers": [
    {},
    {
      "type": "RTSP source",
      "url": "rtsp://localhost:8554/stream1",
      "remote_addr": "[::1]:65513",
      "user_agent": "Lavf59.27.100",
      "medias": [
        "video, sendonly, 96 H265/90000",
        "audio, sendonly, 97 OPUS/48000/2"
      ],
      "tracks": [
        "96 H265/90000, sinks=1",
        "97 OPUS/48000/2, sinks=1"
      ],
      "recv": 6700364
    }
  ],
  "consumers": [
    {
      "type": "WebRTC client",
      "remote_addr": "udp4 prflx 192.168.2.10:60248 related :0",
      "user_agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/16.3 Safari/605.1.15",
      "send": 2613086
    }
  ]
}

Since there is no opus, I transcoded this file to an mp4 following your recommendations using libx265 and hevc_nvenc. However, using a standard FFmpeg build, I could not find a high profile for libx265 and hevc_nvenc and libx265. Could this be the problem? The result was the same for this file's libx265 and hevc_nvenc versions. In addition, audio only played for WebRTC, and only a single frame was shown with no sound for the MSE stream. Please provide me with some guidance on what I might be doing wrong. Thank you for your attention to this matter.

Thank you in advance!

AlexxIT / Blog

Issue with RTSP streaming of x265 or hevc_nvenc encoded file with opus audio: Only audio for WebRTC played and only the first picture for MSE shown. #10