H265 with flag short_term_ref_pic_set_sps_flag cannot play

[bryandario8]

Which version are you using?

v1.1.0

Which operating system are you using?

• [ ] Linux amd64 standard
• [x] Linux amd64 Docker
• [ ] Linux arm64 standard
• [ ] Linux arm64 Docker
• [ ] Linux arm7 standard
• [ ] Linux arm7 Docker
• [ ] Linux arm6 standard
• [ ] Linux arm6 Docker
• [ ] Windows amd64 standard
• [ ] Windows amd64 Docker (WSL backend)
• [ ] macOS amd64 standard
• [ ] macOS amd64 Docker
• [ ] Other (please describe)

Describe the issue

When I consume a camera stream by hls, I get the following error: destroyed (muxer error: unable to extract DTS: short_term_ref_pic_set_sps_flag = true is not supported). So the HLS in the browser doesn't play and I get levelEmptyError: No Segments found in Playlist

I see this error happening here https://github.com/bluenviron/mediacommon/blob/main/pkg/codecs/h265/dts_extractor.go#L82

if shortTermRefPicSetSpsFlag { return 0, fmt.Errorf("short_term_ref_pic_set_sps_flag = true is not supported") }

What does this restriction mean in DST Extractor?

Describe how to replicate the issue

1. start the rtsp proxy from docker
2. publish camera rtsp on main stream with H265
3. read camera from browser Chrome/Mozilla

Did you attach the server logs?

rtsp-proxy-dev-rtsp-proxy-1 | 2023/09/25 23:21:27 INF [HLS] [muxer 10_245_111_122] created (requested by 10.184.32.134) rtsp-proxy-dev-rtsp-proxy-1 | 2023/09/25 23:21:27 INF [HLS] [muxer 10_245_111_122] is converting into HLS, 1 track (H265) rtsp-proxy-dev-rtsp-proxy-1 | 2023/09/25 23:21:29 INF [HLS] [muxer 10_245_111_122] destroyed (muxer error: unable to extract DTS: short_term_ref_pic_set_sps_flag = true is not supported) rtsp-proxy-dev-rtsp-proxy-1 | 2023/09/25 23:21:30 INF [HLS] [muxer 10_245_111_122] created (requested by 10.184.32.134) rtsp-proxy-dev-rtsp-proxy-1 | 2023/09/25 23:21:30 INF [HLS] [muxer 10_245_111_122] is converting into HLS, 1 track (H265) rtsp-proxy-dev-rtsp-proxy-1 | 2023/09/25 23:21:32 INF [HLS] [muxer 10_245_111_122] destroyed (muxer error: unable to extract DTS: short_term_ref_pic_set_sps_flag = true is not supported) rtsp-proxy-dev-rtsp-proxy-1 | 2023/09/25 23:21:34 INF [HLS] [muxer 10_245_111_122] created (requested by 10.184.32.134) rtsp-proxy-dev-rtsp-proxy-1 | 2023/09/25 23:21:34 INF [HLS] [muxer 10_245_111_122] is converting into HLS, 1 track (H265) rtsp-proxy-dev-rtsp-proxy-1 | 2023/09/25 23:21:34 INF [HLS] [muxer 10_245_111_122] destroyed (muxer error: unable to extract DTS: short_term_ref_pic_set_sps_flag = true is not supported) rtsp-proxy-dev-rtsp-proxy-1 | 2023/09/25 23:21:36 INF [HLS] [muxer 10_245_111_122] created (requested by 10.184.32.134) rtsp-proxy-dev-rtsp-proxy-1 | 2023/09/25 23:21:36 INF [HLS] [muxer 10_245_111_122] is converting into HLS, 1 track (H265) rtsp-proxy-dev-rtsp-proxy-1 | 2023/09/25 23:21:37 INF [HLS] [muxer 10_245_111_122] destroyed (muxer error: unable to extract DTS: short_term_ref_pic_set_sps_flag = true is not supported) rtsp-proxy-dev-rtsp-proxy-1 | 2023/09/25 23:21:38 INF [HLS] [muxer 10_245_111_122] created (requested by 10.184.32.134) rtsp-proxy-dev-rtsp-proxy-1 | 2023/09/25 23:21:38 INF [HLS] [muxer 10_245_111_122] is converting into HLS, 1 track (H265) rtsp-proxy-dev-rtsp-proxy-1 | 2023/09/25 23:21:39 INF [HLS] [muxer 10_245_111_122] destroyed (muxer error: unable to extract DTS: short_term_ref_pic_set_sps_flag = true is not supported) rtsp-proxy-dev-rtsp-proxy-1 | 2023/09/25 23:21:41 INF [HLS] [muxer 10_245_111_122] created (requested by 10.184.32.134) rtsp-proxy-dev-rtsp-proxy-1 | 2023/09/25 23:21:41 INF [HLS] [muxer 10_245_111_122] is converting into HLS, 1 track (H265) rtsp-proxy-dev-rtsp-proxy-1 | 2023/09/25 23:21:42 INF [HLS] [muxer 10_245_111_122] destroyed (muxer error: unable to extract DTS: short_term_ref_pic_set_sps_flag = true is not supported)

Did you attach a network dump?

no

[aler9]

Hello, the reason why short_term_ref_pic_set_sps_flag is not supported is just that we've never encountered a H265 with that flag yet. Please attach a network dump, that can be generated in this way:

1) Download wireshark (https://www.wireshark.org/) 2) Start capturing on the interface used for exchanging RTSP (if the server and the target software are both installed on your pc, the interface is probably "loopback", otherwise it's the one of your network card) 3) Start the server and replicate the issue 4) Stop capturing, save the result in .pcap format 5) Attach

[bryandario8]

Hello, thanks for your answer 👍

I followed the instructions and generated the .pcap file, which I attach here

rtsp-proxy.zip

I hope for your response!

[bryandario8]

Hello @aler9, please your help with this review. I need a solution to be able to view the streaming in realtime without error.

[aler9]

Hello, i checked the pcap but unfortunately it doesn't contain the RTSP handshake - it was apparently generated when the stream was already running. We need a pcap that contains the full interaction between the server and the camera, from the moment the server is started to the moment the error appears.

[bryandario8]

Sorry, I didn't take the logs from the start. This time I followed your recommendation, thank you. rtsp-proxy2.zip

[aler9]

Thanks for providing all the material that allowed to successfully replicate the bug. Please test this nightly release (that includes https://github.com/bluenviron/mediacommon/pull/95) and let me know whether it fixes the issue.

[link removed]

[bryandario8]

Hi, I confirm that the release fixes the issue!! Thank you very much @aler9 Please, tell me when the official release will be

[aler9]

added in v1.2.0

[github-actions[bot]]

This issue is being locked automatically because it has been closed for more than 6 months. Please open a new issue in case you encounter a similar problem.