Can't play stream with unknown framerate

[KeloCube]

I'm trying to stream via RTSP from an Android phone using this app: https://play.google.com/store/apps/details?id=com.miv.rtspcamera. The app allows configuring a constant framerate but this is not communicated over the stream. If played with ffplay, the framerate is guessed and the stream plays fine. In CasparCG, however, the framerate is detected as 0, and the stream fails to play.

This can be fixed with a change to modules/ffmpeg/producer/util/util.cpp. The read_framerate function tries to match an fps of 0 with the closest fps it can find from the list of known video formats, and ends up matching it with video_format::invalid, instead of defaulting to fail_value (which would be the channel framerate, as passed by the constructor of ffmpeg_producer). Is this intentional? A sanity check after finding the closest match, or an exclusion of video_format::invalid from the search, would fix the issue.

if (is_sane_fps(av_make_q(closest_fps.numerator(), closest_fps.denominator())))
    return closest_fps;

Would adding this break something I'm not aware of? If not, I can submit a PR for it.

Having ffmpeg actually guess the framerate might be another fix, but seems harder to implement since ffmpeg doesn't seem to probe for the fps immediately on the call to avformat_open_input or avformat_find_stream_info. This wouldn't be a great fix for CasparCG anyway, since the guessed framerate is usually not exact.

[Punkley]

I actually saw this while playing the other day.

i was trying to re-stream https twitch, if the framerate returned 0 it failed. I woud be interested in testing this.

[KeloCube]

Sure, here's a Windows build: https://1drv.ms/u/s!ABScl-crjvcacg

This build also contains another patch I had to make to prevent the stream from getting stuck and eventually overflowing if only the video or only the audio stream receives a flush message, but the other doesn't. I might report that issue separately later, if this one is recognized.

I'm not sure whether these types of fixes are welcome in the first place, since playing streams is not what the ffmpeg producer (or CasparCG, for that matter) was really built for. The only good solution to playing streams would be to modify or rewrite the producer to use presentation timestamps, since you can't rely on a network stream to have a constant framerate or to even be able to provide data continously.

[Punkley]

I'm compiling my own version, adding a couple of other fixes I need, where did you add those lines in

[KeloCube]

Here's a patch file for both this fix and the other one I mentioned. Or you could just pull the changes from https://github.com/KeloCube/Server/tree/streaming-fix and https://github.com/KeloCube/Server/tree/audio-video-flush-fix.

audio-video-flush-fix.diff.txt streaming-fix.diff.txt

[Punkley]

Hmm try this,

go to twitch.TV stream, change the stream bitrate to 480p or something other then source, then inspect the network in the browser to get the m3u8 file,

go to CasperCG do "PLAY 1-1 url_ending_in.m3u8" for me the who screen just crashes with an error message, before the changes for me it just wouldn't play saying the stream was 0 bitrate.

[KeloCube]

I tried this but couldn't get it to work either. Judging by the error messages (OpenGL errors and an assert failure), it would seem like CasparCG is receiving some frames with no data or invalid data, but I'm not sure about this.

Twitch streams are working fine when played with ffplay, btw. One thing I did notice, however, was that the stream was stuttering quite a bit when played with the ffmpeg binaries included with the CasparCG 2.1.0 branch (look in dependencies64\ffmpeg\bin\win32), whereas playback was smooth with the latest Windows build of ffmpeg. So updating to a later version of ffmpeg might fix the issue, but I wouldn't get my hopes up. Fix or not, you'd have to do it anyway if you wanted to play the stream directly in CasparCG, because a stuttering stream is pretty useless.

Your best bet would probably be to try one of the following:

• Play the stream in a html or flash template
• Restream the Twitch stream as RTSP with e.g. VLC, and then receive the RTSP stream in CasparCG.
• Use some software that can capture your screen as a DirectShow device (XSplit comes to mind).

I would have used one of these strategies too, were it not for the fact that I need to run multiple RTSP cameras simultaneously, and permormance issues arise when using something like this for more than one or two streams.

[Julusian]

I'm not sure whether these types of fixes are welcome in the first place, since playing streams is not what the ffmpeg producer (or CasparCG, for that matter) was really built for.

I think that fixes to stream playback would be accepted as work has already been done to allow CasparCG to play streams. Plus, some people are interested in running CasparCG in the cloud, and they will need stream playback to be robust.

The only good solution to playing streams would be to modify or rewrite the producer to use presentation timestamps.

I have seen this same idea in a few other issues, as there are issues with seeking and audio sync that would be solved by using packet timestamps.

[KeloCube]

Alright, thanks. I'm just a little hesistant since this could technically break something if video_mode::invalid was the expected return value from read_framerate (instead of fail_value).

By "whether these types of fixes are welcome or not", I meant that as professional production software, CasparCG might want to prioritize "stability over ability" when it comes to playing streams, for example. On the one hand, this is a small fix for a rather niche use case, but on the other, I think this particular fix is unlikely to break anything. Even if this fix isn't merged into the current dev branch, I still think it's valuable to have the issue and fix documented (either here or on the forums).

I'd love to hear some input from the maintainers of the project, too!

[ronag]

@KeloCube please test with 2.2 and reopen if still an issue.

[KeloCube]

I actually tried this as soon as I saw the 2.2 builds. All my streams play without issue now, just as they would using ffplay. Thank you for the ffmpeg producer rewrite, it has been a very welcome improvement!

The only stream I couldn't get to play was the twitch stream. I tested on the last Windows binary you linked to in #679, as I had some issues with the build and didn't want to get into resolving those at the moment.

I tracked down a playlist url for a 24/7 stream for testing, however: https://video-weaver.arn03.hls.ttvnw.net/v1/playlist/CvUCecZW-b2fnk-ww7BqvO3ac7tcwCOzIv83GoTHcACgYAz-DP8_qwZAvGV_MOmnCp7VpIoXByhlT3OAwrYVjteV1G4oWsyeLjblTZgQSqlpv-lnIPGehORKTAOsm_lzManczinGBHfaHtRbAWbHejkptQnz1DsBmdwYl-pbB661Cuxx2fkqpMT-Mb2SV7WR1TPppnQGxCctpmfBZR_mjYw9PAMitEfG-52sW3UMsN2r_-VFGmQpheO19H61jDzPeBDTPEwjXf_IH3tG_35lgLT-bc3NT7mGaStg7cgD3Fq0QJo-8cswMcsi1vzHhOfE_kkGAkx-xpiaIa-qTMMlp63fVNOWLKshlY-ABxT2rjSyDd4xv_EVk2MEZaqNPlW-aH4QO0ZwTmWnj8dzVsGpJOa_N9OAnuoB9yxKyu7xQekmclggQnusBWT7CqplEW73IiIQSOQsdQEsv7a7q0Pa2LUH269AhW9HXhhJ0OXklfhqZgCccGBzRxIQMcVD8Smq_oiPQDdcVHQ8BRoMEkWRpNTdurxU-njm.m3u8

Again, this url plays fine using ffplay, but not in CasparCG. Or at least not the build I used...

Reopen this if you can confirm that the stream doesn't work and it's something you want to work on.

[ronag]

@KeloCube that stream has some wierd timestamp. Not sure we can fix it. CasparCG makes a bit more strict assumption about the input than ffplay does.

Could you open this as a separate issue and we can look into it at some point in the future. Thanks!