Closed bradenfallon closed 1 year ago
Hi. Thanks for reporting and sorry for the delay. Could these errors be legit? If the connection is broken while sending data, these would be expected logs.
No worries about the delay. These errors are legit, as the Youtube live stream freezes about a day after I start the broadcast. The stream does not end, indicating that data is still being received but is stuck in a buffer loop.
I cannot identify why the connection is being closed/broken and why the stream freezes. My thought is it is an issue with the FFMPEG encoding getting stuck.
It has reached the point where the stream freezes nearly every day and sometimes multiple times per day. https://www.youtube.com/watch?v=KlaXNkelQ78
Okay thanks. What I meant by legit is, should we assume that the connection is broken because of a connectivity issue with the machine running the script? In which case, we should make sure the stream does restart properly.
Or, else, do you believe that this is not caused by actual network issues?
Unfortunately, I have been trying to ascertain whether it is an issue with the network or not. I suspect it is an issue with FFMPEG since the connection error specifically occurs when the video file ends and is decoded and prepared again. However, it does not happen every time the video file loops.
Ok I think I know what this is. The fallible=true
is causing you issues. When the file ends, the source is momentarily down, which causes the output to stop the connection and try to restart it. However, the server on the other side is not ready to restart streaming.
You should wrap your source in a mksafe()
and remove fallible=true
. I believe that this will solve your issue.
@toots,
I appreciate all of your help!
I tried wrapping my video source in mksafe()
using the following command: video_file = mksafe(video_file)
.
However, upon trying to save that configuration I get the following error:
LogicException at /var/azuracast/www/src/Radio/Backend/Liquidsoap.php L246: At line 321, char 20-30: Error 5: this value has type string (inferred at line 320, char 13-68) but it should be a subtype of the type of the value at /usr/share/liquidsoap/2.1.4/libs/source.liq, line 5, char 24 source(_)
Being that I don't fully understand what mksafe()
does, I wonder if it is conflicting with the command single()
or video.add_text.ffmpeg.raw()
.
@toots,
I appreciate all of your help!
I tried wrapping my video source in
mksafe()
using the following command:video_file = mksafe(video_file)
. However, upon trying to save that configuration I get the following error:LogicException at /var/azuracast/www/src/Radio/Backend/Liquidsoap.php L246: At line 321, char 20-30: Error 5: this value has type string (inferred at line 320, char 13-68) but it should be a subtype of the type of the value at /usr/share/liquidsoap/2.1.4/libs/source.liq, line 5, char 24 source(_)
Being that I don't fully understand what
mksafe()
does, I wonder if it is conflicting with the commandsingle()
orvideo.add_text.ffmpeg.raw()
.
mksafe
needs to be applied on the last source before your output:
EDITED: won't work, see below!
Ha but you're using the raw
format so this won't work.
The most convenient solution for you would be to create a blank file with the same video format as your output and add it as a security fallback:
# VIDEO STREAM
settings.decoder.ffmpeg.codecs.vp9.set("libvpx-vp9")
# Station Base Directory
station_base_dir = "/var/azuracast/stations/chillradio"
# RTMP Stream URL (YouTube)
url = "rtmp://a.rtmp.youtube.com/live2/[REDACTED]"
# Defining video file
video_file = station_base_dir ^ "/media/videostream/Hipster-Cat.mp4"
# Defining the Logo File
logo_file = station_base_dir ^ "/media/videostream/Hipster-Cat-Banner2.png"
# Defining the font for Now Playing
font_file = station_base_dir ^ "/media/videostream/bebasneue.otf"
# Align text and define font size
font_size = 55
font_x = 75
font_y = 875
# Add Now Playing metadata
nowplaying = ref("")
def update_nowplaying(m)
nowplaying := m["artist"] ^ "\n" ^ "------------------------" ^ "\n" ^ "\"" ^ m["title"] ^ "\""
end
radio.on_metadata(update_nowplaying)
# Method to overlay Now Playing text
def add_nowplaying_text(d, s) =
def mkfilter(graph)
v = video.add_text.ffmpeg.raw(font=font_file,size=font_size,speed=0,x=font_x,y=font_y,graph=graph,d,s)
ffmpeg.filter.video.output(graph, v)
end
ffmpeg.filter.create(mkfilter)
end
# Method to create image overlay
def add_logo(background, logo) =
def mkfilter(logo_plot)
background = ffmpeg.filter.video.input(logo_plot, background)
logo = ffmpeg.filter.video.input(logo_plot, logo)
logo = ffmpeg.filter.scale(logo_plot, logo, w="-1", h="178")
logo_add = ffmpeg.filter.overlay(logo_plot, background, logo, y="0", x="1780")
ffmpeg.filter.video.output(logo_plot, logo_add)
end
ffmpeg.filter.create(mkfilter)
end
# Muxing the text and video streams
logo = single(logo_file)
videostream = single(video_file)
videostream = add_nowplaying_text({!nowplaying}, videostream)
videostream = add_logo(videostream, logo)
# Add the audio stream
videostream = mux_video(video=videostream, radio)
# Add a security blank file
videostream = fallback(track_sensitive=false, [
single("/path/to/blank.mp4"),
videostream
])
# Output to YouTube RTMP
enc = %ffmpeg(
format="flv",
%video.raw(codec="libx264", pixel_format="yuv420p", bufsize="8000k", b="8000k", minrate="4000k", maxrate="10000k", preset="faster", r=30, g=60),
%audio(codec="aac", samplerate=44100, b="128k")
)
# Broadcasting RTMP
output.url(url=url, enc, videostream)
With this your source will never become unavailable and disconnect the output!
@toots
I managed to get the stream working by changing this line and creating a copy of my background video versus a blank video:
# Add a security blank file
videostream = fallback(track_sensitive=false, [
videostream,
single("/path/to/videostream_copy.mp4"),
])
I will monitor it to see if anything odd still happens. I do get a few errors in the log.
2023/03/26 00:31:06 [lang:3] API feedback - Response (200): true
2023/03/26 00:31:06 [clock.main:2] We must catchup 1.00 seconds!
[libx264 @ 0xffff6cc0fd50] invalid DTS: PTS is less than DTS
[libx264 @ 0xffff6cc0fd50] VBV underflow (frame 41, -4671459 bits)
[flv @ 0xffff6c0a1be0] pts (67) < dts (1300) in stream 1
[libx264 @ 0xffff6cc0fd50] invalid DTS: PTS is less than DTS
[flv @ 0xffff6c0a1be0] pts (100) < dts (1333) in stream 1
2023/03/26 00:31:07 [output.url:2] Error while encoding data: Avutil.Error(Invalid argument)
2023/03/26 00:31:07 [output.url:3] Will try again in 2.00 seconds.
[libx264 @ 0xffff5c0b84c0] specified frame type (3) at 0 is not compatible with keyframe interval
Glad you were able to make it work! These logs look concerning, still, you should have a consistent DTS for your whole stream. Can you post the current script? I might try to reproduce locally.
@toots
Here is the current script I am running:
# VIDEO STREAM
settings.decoder.ffmpeg.codecs.vp9.set("libvpx-vp9")
# Station Base Directory
station_base_dir = "/var/azuracast/stations/chillradio"
# RTMP Stream URL (YouTube)
url = "rtmp://a.rtmp.youtube.com/live2/[Stream_Key]"
# Defining video file
video_file = station_base_dir ^ "/media/videostream/Hipster-Cat.mp4"
blank_file = station_base_dir ^ "/media/videostream/Hipster-Cat-Backup.mp4"
# Defining the Logo File
logo_file = station_base_dir ^ "/media/videostream/Hipster-Cat-Banner2.png"
# Defining the font for Now Playing
font_file = station_base_dir ^ "/media/videostream/bebasneue.otf"
# Align text and define font size
font_size = 55
font_x = 75
font_y = 875
# Add Now Playing metadata
nowplaying = ref("")
def update_nowplaying(m)
nowplaying := m["artist"] ^ "\n" ^ "------------------------" ^ "\n" ^ "\"" ^ m["title"] ^ "\""
end
radio.on_metadata(update_nowplaying)
# Method to overlay Now Playing text
def add_nowplaying_text(d, s) =
def mkfilter(graph)
v = video.add_text.ffmpeg.raw(font=font_file,size=font_size,speed=0,x=font_x,y=font_y,graph=graph,d,s)
ffmpeg.filter.video.output(graph, v)
end
ffmpeg.filter.create(mkfilter)
end
# Method to create image overlay
def add_logo(background, logo) =
def mkfilter(logo_plot)
background = ffmpeg.filter.video.input(logo_plot, background)
logo = ffmpeg.filter.video.input(logo_plot, logo)
logo = ffmpeg.filter.scale(logo_plot, logo, w="-1", h="178")
logo_add = ffmpeg.filter.overlay(logo_plot, background, logo, y="0", x="1780")
ffmpeg.filter.video.output(logo_plot, logo_add)
end
ffmpeg.filter.create(mkfilter)
end
# Muxing the text and video streams
logo = single(logo_file)
videostream = single(video_file)
videostream = add_nowplaying_text({!nowplaying}, videostream)
videostream = add_logo(videostream, logo)
# Add the audio stream
videostream = mux_video(video=videostream, radio)
# Add a security blank file
videostream = fallback(track_sensitive=false, [
videostream,
single(blank_file)
])
# Output to YouTube RTMP
enc = %ffmpeg(
format="flv",
%video.raw(codec="libx264", pixel_format="yuv420p", bufsize="8000k", b="8000k", minrate="4000k", maxrate="10000k", preset="faster", r=30, g=60),
%audio(codec="aac", samplerate=44100, b="128k")
)
# Broadcasting RTMP
output.url(url=url, enc, videostream)
Also, here is the full log I just exported. It looks like the DTS/PTS issues only occur during the initial loading of the backup file. It seems that everything is running well now.
I can also give you access to my Azuracast instance if that would be helpful. Just shoot me an email at braden.fallon@gmail.com if you'd like that.
I can confirm the issue and have traced it down, will report soon. Thanks!
The issue with switching raw
ffmpeg format is now fixed in main
, rolling-release-v2.2.x and the brand new rolling-release-v2.1.x
I'm not sure if we'll do another 2.1.x
release, for now this bugfix is probably not enough to justify it. However, I believe that azuracast supports rolling releases so they might be able to allow you to use it.
I've tested your script with both 2.1.x
and 2.2.x
syntax. Here they are:
I forgot to say: in the script rewrite, I have condensed all use of ffmpeg filter in a single operator. This is a better way to optimize and avoid some of the complications linked with ffmpeg filters support.
@toots
Buster updated Azuracast to use the latest 2.1.X Rolling Release of Liquidsoap. I plugged in your script and a few issues arose.
First, the line radio = playlist(station_base_dir ^ "/playlist")
did not exist. I think that is how Azuracast processes the radio.
Commenting out that line fixed that issue.
Second, the consolidated filter is half working. The logo appears on the livestream but the now playing metadata is no longer showing. Looking at the code, it seems like that should be working, although I see a new function called video.add_text.ffmpeg.raw.filter()
. Here is a link to the stream so you can see: https://youtube.com/live/wstPyjwWIag
2023/03/28 13:24:34 [clock:3] Using native (high-precision) implementation for latency control
2023/03/28 13:24:37 [frame:3] Using 44100Hz audio, 30Hz video, 44100Hz main.
2023/03/28 13:24:37 [frame:3] Video frame size set to: 1920x1080
2023/03/28 13:24:37 [frame:3] Frame size must be a multiple of 1470 ticks = 1470 audio samples = 1 video samples.
2023/03/28 13:24:37 [frame:3] Targeting 'frame.duration': 0.04s = 1764 audio samples = 1764 ticks.
2023/03/28 13:24:37 [frame:3] Frames last 0.07s = 2940 audio samples = 2 video samples = 2940 ticks.
2023/03/28 13:24:37 [sandbox:3] Running inside a docker container, disabling sandboxing.
2023/03/28 13:24:37 [video.converter:3] Using preferred video converter: ffmpeg.
2023/03/28 13:24:37 [audio.converter:3] Using samplerate converter: libsamplerate.
2023/03/28 13:24:37 [error_jingle:3] "/usr/local/share/icecast/web/error.mp3" is static, resolving once for all...
2023/03/28 13:24:38 [error_jingle:3] Prepared "/usr/local/share/icecast/web/error.mp3" (RID 2).
2023/03/28 13:24:38 [single_3:3] "/var/azuracast/stations/chillradio/media/videostream/Hipster-Cat.mp4" is static, resolving once for all...
2023/03/28 13:24:39 [single_3:3] Prepared "/var/azuracast/stations/chillradio/media/videostream/Hipster-Cat.mp4" (RID 5).
2023/03/28 13:24:40 [clock.main:3] Streaming loop starts in auto-sync mode
2023/03/28 13:24:40 [clock.main:3] Delegating synchronisation to CPU clock
2023/03/28 13:24:40 [single_2:3] "/var/azuracast/stations/chillradio/media/videostream/Hipster-Cat.mp4" is static, resolving once for all...
2023/03/28 13:24:40 [single_2:3] Prepared "/var/azuracast/stations/chillradio/media/videostream/Hipster-Cat.mp4" (RID 4).
2023/03/28 13:24:40 [source:1] Failed to prepare track: request not ready.
2023/03/28 13:24:40 [switch_7:3] Switch to mux_audio_1.
2023/03/28 13:24:40 [source:1] Failed to prepare track: request not ready.
2023/03/28 13:24:40 [safe_fallback:3] Switch to metadata.map_1.
[mp3float @ 0xffff6cf44280] Could not update timestamps for skipped samples.
2023/03/28 13:24:40 [local_1:3] Connecting mount /radio-128.mp3 for source@127.0.0.1...
2023/03/28 13:24:40 [local_1:3] Connection setup was successful.
2023/03/28 13:24:40 [local_2:3] Connecting mount /radio-256.mp3 for source@127.0.0.1...
2023/03/28 13:24:40 [local_2:3] Connection setup was successful.
[mpegts @ 0xaaab0bf6bec0] frame size not set
[mpegts @ 0xaaab0bdc20c0] frame size not set
[mpegts @ 0xaaab0bf46da0] frame size not set
[mpegts @ 0xaaab0bf75c70] frame size not set
2023/03/28 13:24:40 [source:1] Failed to prepare track: request not ready.
2023/03/28 13:24:40 [source:1] Failed to prepare track: request not ready.
2023/03/28 13:24:40 [source:1] Failed to prepare track: request not ready.
2023/03/28 13:24:41 [source:1] Failed to prepare track: request not ready.
2023/03/28 13:24:41 [source:1] Failed to prepare track: request not ready.
2023/03/28 13:24:41 [source:1] Failed to prepare track: request not ready.
2023/03/28 13:24:41 [source:1] Failed to prepare track: request not ready.
2023/03/28 13:24:41 [source:1] Failed to prepare track: request not ready.
2023/03/28 13:24:41 [source:1] Failed to prepare track: request not ready.
2023/03/28 13:24:41 [source:1] Failed to prepare track: request not ready.
2023/03/28 13:24:41 [source:1] Failed to prepare track: request not ready.
2023/03/28 13:24:41 [single_1:3] "/var/azuracast/stations/chillradio/media/videostream/Hipster-Cat-Banner2.png" is static, resolving once for all...
2023/03/28 13:24:41 [single_1:3] Prepared "/var/azuracast/stations/chillradio/media/videostream/Hipster-Cat-Banner2.png" (RID 3).
2023/03/28 13:24:41 [switch_7:3] Switch to mux_audio_0 with transition.
2023/03/28 13:24:41 [video_main_0:2] Warning: there may be an infinite sequence of empty tracks!
2023/03/28 13:24:41 [video_main_0:2] Warning: there may be an infinite sequence of empty tracks!
2023/03/28 13:24:41 [video.text:3] Using camlimages implementation
2023/03/28 13:24:41 [lang:3] API nextsong - Sending POST request to 'http://127.0.0.1:6010/api/internal/1/liquidsoap/nextsong' with body:
2023/03/28 13:24:41 [lang:3] API nextsong - Response (200): annotate:title="40g",artist="Puar, Bionsen",duration="91.00",song_id="8b1e1629f216ae7804014deb10009572",media_id="175",liq_amplify="0.00dB",liq_cross_duration="0.00",liq_fade_in="0.00",liq_fade_out="0.00",playlist_id="2":media:Chilled Cat - Lofi Non-Copyright/puar,_bionsen_-_40g.mp3
2023/03/28 13:24:42 [next_song:3] Prepared "/var/azuracast/stations/chillradio/media/Chilled Cat - Lofi Non-Copyright/puar,_bionsen_-_40g.mp3" (RID 7).
2023/03/28 13:24:42 [lang:3] AutoDJ is ready!
2023/03/28 13:24:42 [output.url:2] Warning: there may be an infinite sequence of empty tracks!
2023/03/28 13:24:42 [output.url:2] Warning: there may be an infinite sequence of empty tracks!
2023/03/28 13:24:42 [output.url:2] Warning: there may be an infinite sequence of empty tracks!
2023/03/28 13:24:42 [clock.main:2] We must catchup 1.72 seconds!
2023/03/28 13:24:42 [lang:3] API nextsong - Sending POST request to 'http://127.0.0.1:6010/api/internal/1/liquidsoap/nextsong' with body:
2023/03/28 13:24:42 [safe_fallback:3] Switch to amplify_3 with transition.
2023/03/28 13:24:42 [lang:3] API nextsong - Response (200): annotate:title="Lonely Forest",artist="Wood.",duration="129.00",song_id="e25b40535e4599432ad886c6d19a4ece",media_id="197",liq_amplify="0.00dB",liq_cross_duration="0.00",liq_fade_in="0.00",liq_fade_out="0.00",playlist_id="2":media:Chilled Cat - Lofi Non-Copyright/wood._-_lonely_forest.mp3
2023/03/28 13:24:42 [interrupting_fallback:3] Switch to requests_fallback.
2023/03/28 13:24:42 [requests_fallback:3] Switch to autodj_fallback.
2023/03/28 13:24:42 [autodj_fallback:3] Switch to dynamic_startup.
2023/03/28 13:24:42 [dynamic_startup:3] Switch to cue_next_song.
[mp3float @ 0xffff6cb51300] Could not update timestamps for skipped samples.
2023/03/28 13:24:42 [cue_next_song:3] Cueing in...
2023/03/28 13:24:42 [lang:3] API feedback - Sending POST request to 'http://127.0.0.1:6010/api/internal/1/liquidsoap/feedback' with body: {
2023/03/28 13:24:42 [lang:3] "song_id": "8b1e1629f216ae7804014deb10009572",
2023/03/28 13:24:42 [lang:3] "playlist_id": "2",
2023/03/28 13:24:42 [lang:3] "media_id": "175"
2023/03/28 13:24:42 [lang:3] }
2023/03/28 13:24:42 [lang:3] API feedback - Response (200): true
Okay. You can revert to your script first and see if that works. The fix should be good for it regardless. My version was a cleanup that is optional.
I'll try reverting back just the filter portion of the script. I was trying to figure out why one part of the filter would work and the other wouldn't.
I greatly appreciate all your help and was certainly not expecting this level of assistance!
There may be an issue with the function video.add_text.ffmpeg.raw.filter(). The function seems to process static text perfectly but is exhibiting issues when a function or dynamic text is used as input.
Describing the Issue I am running an online radio using Azuracast. However, since this error appears to be part of the Liquidsoap component, I am putting it here.
I have been running into an error where after running my Youtube Livestream for more than a couple of days, the stream will freeze and continually buffer. Once I restart the broadcast, the livestream resumes, and all is well for a few days. When the freeze occurs, the backend radio system is still running fine, but I see this error in my Liquidsoap logs. It appears to occur right after loading the video for the stream, but not every instance of loading the video.
Here is another instance of the error.
Here is the script that runs the livestream broadcast:
Version details OS: Ubuntu 22.04 server on Oracle Cloud Version: Liquidsoap 2.1.4 is currently used.
Install method AzuraCast Rolling Release #[4cd3f65], Docker, PHP 8.2