savonet / liquidsoap

Liquidsoap is a statically typed scripting general-purpose language with dedicated operators and backend for all thing media, streaming, file generation, automation, HTTP backend and more.
http://liquidsoap.info
GNU General Public License v2.0
1.4k stars 128 forks source link

How to make prepend work as expected? → Avoid "Candidate to prepending not ready. Abort!" #240

Closed Moonbase59 closed 3 years ago

Moonbase59 commented 9 years ago

Hi, I'm still working on my radio script so many hours a day—Liquidsoap ain't as easy as I thought!

Our station gets introductory IDs from various bands, which shall be played just before one of their songs, often album-specific. So I came up with idea to name those according to an »Artist - Album« scheme in a separate »ids« folder like this:

AC_DC.mp3 (can be played before any AC/DC song)
Edguy - Tinnitus Sanctus.mp3 (shall be played before any song from the album »Tinnitus Sanctus«)

These ID files cannot be prepared beforehand, because LS runs 24/7 and sometimes the file from a band comes quite late, and just gets uploaded via FTPES into the special »ids« folder. Also, the playlist is generated randomly from whatever is currently uploaded into the filesystem.

So we really need to insert a band ID on-the-fly, depending on whatever the next song to be played has in it's metadata.

Now, for prepend, the docs state something like »the source to be prepended must be immediately ready« and I actually get this error:

2015/07/24 16:18:33 [prepend_5080:3] Candidate to prepending not ready. Abort!

Has anyone please got an idea how prepend can be made to work? Or an easy way to circumvent this problem?

The logs show that the »file finding« works ok:

2015/07/24 16:12:49 [tagged:3] Prepared "/home/matthias/music/tagged/Edguy/Edguy - Burning Down the Opera_ Live/Edguy - Land of the Miracle.mp3" (RID 6).
2015/07/24 16:12:49 [build_id_source:3] Looking for "/home/matthias/music/ids/Edguy - Burning Down the Opera_ Live.mp3"
2015/07/24 16:12:49 [build_id_source:3] Looking for "/home/matthias/music/ids/Edguy.mp3"
2015/07/24 16:12:49 [build_id_source:3] Found Artist ID "/home/matthias/music/ids/Edguy.mp3"
2015/07/24 16:12:49 [single:3] "annotate:type=\"jingle\":/home/matthias/music/ids/Edguy.mp3" will be queued.
2015/07/24 16:12:49 [prepend_5080:3] Candidate to prepending not ready. Abort!
2015/07/24 16:12:49 [decoder:3] Method "MAD" accepted "/home/matthias/music/ids/Edguy.mp3".
2015/07/24 16:12:49 [amplify_5082:3] Overriding amplification: 0.422669.
2015/07/24 16:18:22 [decoder:3] Method "MAD" accepted "/home/matthias/music/tagged/Accept/Accept - Accept/Accept - Seawinds.mp3".
2015/07/24 16:18:33 [tagged:3] Finished with "/home/matthias/music/tagged/Edguy/Edguy - Burning Down the Opera_ Live/Edguy - Land of the Miracle.mp3".
2015/07/24 16:18:33 [amplify_5082:3] End of the current overriding.
2015/07/24 16:18:33 [tagged:3] Prepared "/home/matthias/music/tagged/Accept/Accept - Accept/Accept - Seawinds.mp3" (RID 1).
2015/07/24 16:18:33 [build_id_source:3] Looking for "/home/matthias/music/ids/Accept - Accept.mp3"
2015/07/24 16:18:33 [build_id_source:3] Looking for "/home/matthias/music/ids/Accept.mp3"
2015/07/24 16:18:33 [build_id_source:3] Neither Artist+Album nor Artist ID found.
2015/07/24 16:18:33 [prepend_5080:3] Candidate to prepending not ready. Abort!
2015/07/24 16:18:33 [amplify_5082:3] Overriding amplification: 0.970510.
2015/07/24 16:22:54 [decoder:3] Method "MAD" accepted "/home/matthias/music/tagged/Edguy/Edguy - Rocket Ride/Edguy - Wasted Time.mp3".
2015/07/24 16:23:04 [tagged:3] Finished with "/home/matthias/music/tagged/Accept/Accept - Accept/Accept - Seawinds.mp3".

(In this example, the ID file Edguy.mp3 has been found, and we had no ID file(s) for Accept.)

Here is the script code (stripped down to make debugging easier):

#
# radio5.liq - script shortened for bugfixing!
#2015-07-13 v0.2.3 Moonbase
#2015-07-12 v0.2.2 Moonbase
#2015-07-15 v0.2.3 Moonbase
#2015-07-24 v0.2.4 Moonbase
#
# Current issues:
# - Fallback to single emergency file never returns to playlist. What can be done?
#   -> Check Icecast config
#
# - Playlist doesn't seem to be reloaded (in spite of "watch") when files are removed/added
#   within ~/music/tagged or ~/music/jingles.
#
# - Prepending Artist+Album or Artist IDs fails:
#   Candidate to prepending not ready. Abort!
#
# Requirements/Tested with:
# - Ubuntu 14.04.2 LTS
# - Liquidsoap 1.1.1-6ubuntu2
# - Must be kept compatible with Airtime 2.5.2 or later (eventually)
#
# Directory structure:
#   ~/automation    -- This script and the log (radio2.log)
#   ~/stream        -- The fallback emergency file (MP3, 128k CBR, 44.1, 2ch)
# These are reachable via FTPES (for authorized personnel):
#   ~/music/tagged  -- Auto DJ rotation (music files); FLAC, OGG, MP3
#   ~/music/jingles -- Some jingles, mainly sweepers & bumpers used in between as station ids
#   ~/music/promos  -- Show promos for the regular evening shows
#   ~/music/ids     -- Radio IDs we get from the artists, to be played before an appropriate title

# *** Some Settings ***
#
home = "/home/matthias"
#set("log.file.path", home^"/automation/radio5.log")
set("log.file",true)
set("log.stdout",false)
set("log.level",3)

# We use espeak instead of festival for "say:" commands because it supports German more easily.
#set("say.program", home^"/automation/liquidtts")

set("server.telnet",true)

# *** Emergency ***
#
# Emergency fallback - prerecorded single file
# This file is also used by Icecast (in case Liquidsoap fails).
# Must be 128kbps CBR, 44.100Hz, 2ch.
emergency = single(home^"/stream/stream-offline.mp3")

# *** The Songs ***
#
# type="song"
# TODO: Further testing if "watch" works correctly on directory playlist.
songs=playlist(
  home^"/music/tagged",
  reload_mode="watch",
  mode="randomize",
  prefix="annotate:type=\"song\":"
)

# trim silence (below -40 dB) at beginning and end of tracks
s = eat_blank(songs)

# *** Insert artist IDs, if we got any ***
#
# Clean filename (no path)
# extension can be specified but is optional (specify as ".mp3", for instance)
def clean_filename(fn, ~ext="", ~allowtrailingdot=false) =
  # the replacement character
  r = "_"
  fn = string.replace(pattern="[\\*/:<>?|\"]", (fun (s) -> r), string.replace(pattern="^\s+|\s+$", (fun (s) -> ""), fn))
  fn = fn ^ ext
  # Trim again because ext might be bad.
  fn = string.replace(pattern="^\s+|\s+$", (fun (s) -> ""), fn)
  # Windows won't have a "." at end (for folder names)
  if allowtrailingdot == false then
    string.replace(pattern="\.$", (fun (s) -> r), fn)
  else
    fn
  end
end

# Build ID source (a single) from a song's metadata
def build_id_source(m, ~path=home^"/music/ids/") =
  # Only for actual songs, don't build "jingle on jingle"
  if m["type"] == "song" then
    # see if we have an "Artist - Album.mp3" ID file
    fn = path ^ clean_filename(m["artist"] ^ " - " ^ m["album"], ext=".mp3")
    log(label="build_id_source", "Looking for \"" ^ fn ^ "\"")
    if m["artist"] != "" and m["album"] != "" and file.exists(fn) then
      log(label="build_id_source", "Found Album ID \"" ^ fn ^ "\"")
      single("annotate:type=\"jingle\":" ^ fn)
    else
      # see if we have an "Artist.mp3" ID file at least
      fn = path ^ clean_filename(m["artist"], ext=".mp3")
      log(label="build_id_source", "Looking for \"" ^ fn ^ "\"")
      if m["artist"] != "" and file.exists(fn) then
        log(label="build_id_source", "Found Artist ID \"" ^ fn ^ "\"")
        single("annotate:type=\"jingle\":" ^ fn)
      else
        # neither artist+album nor artist ID found
        log(label="build_id_source", "Neither Artist+Album nor Artist ID found.")
        fallback([])
      end
    end
  else
    # not a song
    log(label="build_id_source", "Not type song, ain't looking.")
    fallback([])
  end
end

s = prepend(s, build_id_source(path=home^"/music/ids/"))

# *** Loudness ***
#
# Apply replay gain (all files supposed to have been pre-processed elsewhere)
#   Note: Can't use script at "#{configure.libdir}/extract-replaygain",
#   because it uses mp3gain which in turn messes up files by adding those d**n APE tags!
#   A PLAYING library should *never ever* change the original input data!
# Some "should-bes": +6dB = 2.0, +7dB = 2.24
# TODO: Check if uppercase FLAC/OGG tags "REPLAYGAIN_TRACK_GAIN" are interpreted correctly
# TODO: Check what happens if tag missing
# TODO: Check how amplification works: Is 2.0 actually only +6dB in Liquidsoap? (Use 1. for now.)
s = amplify(override="replaygain_track_gain",1.,s)

# *** Crossfade ***
#
s = smart_crossfade(width=10.,s)

# *** Fallback ***
#
# Make stream "safe" by using a fixed fallback file
s = fallback(track_sensitive=false,[s,emergency])

# *** Output ***
#
# Output stream to local Icecast
# Use a separate clock to allow for some buffering.
clock.assign_new(
  id="icecast",
  [
        output.icecast(
            %mp3(bitrate = 128),
            host = "localhost",
            port = 8000,
            mount = "stream.mp3",
            icy_metadata = "true",
            user = "source",
            password = "hackme",
            name = "Radio Paranoid",
            description = "DEIN Rock- und Metal-Radio! Die Moderatoren live im Chat für Dich.",
            genre = "Rock, Hardrock, Metal",
            url = "http://www.radio-paranoid.de/",
            public = true,
            s
        )
  ]
)
TrurlMcByte commented 9 years ago

in line s = prepend(s, build_id_source(path=home^"/music/ids/")) you don't have "immediately ready" source, because s = eat_blank(songs) before

dbaelde commented 9 years ago

From reading the description of your needs, it's not clear to me that prepend is the right operator. Why not a request.dynamic or a queue of some sort, hooked to some external script of yours dealing with your scheduling logic?

Moonbase59 commented 7 years ago

Sorry for not reacting earlier, so much real life came between me and my hobby radio project …

@TrurlMcByte: leaving out s = eat_blank(songs) didn’t help.

@dbaelde: I seem unable to get this request stuff working. The problem is, that I need the metadata for the upcoming (random) song FIRST so I can cleanup the filename and THEN start looking if an ID file for either the album or the artist even exists.

When using append instead of prepend everything works just fine (except of course the ID plays AFTER the song, and it needs to be played BEFORE the song).

Since all this seems to be a timing issue, I wonder if somehow LS could be made to look just a little earlier so it had time enough to prepare the prepended file … I do know that it starts processing other stuff well before the next song starts.

Moonbase59 commented 5 years ago

For all those search engines (and myself in 5 years …): Here’s a quite short example that works using opam-installed Liquidsoap 1.3.7 (it takes the next song to play from a script):

# test-04.liq: Use kPlaylist as input and prepend IDs

set("log.file.path", "/home/matthias/Musik/liquidsoap/test-04.log")
set("log.file",true)
set("log.stdout",false)
set("log.level",3)
set("server.telnet",true)
set("init.daemon.pidfile.path", "/home/matthias/Musik/liquidsoap/liquidsoap.pid")

def get_next() =
  # Get the first line of my ices/kplaylist script’s output (the URI)
  # (the 2nd line contains an optional stream title)
  lines = get_process_lines("/home/matthias/Musik/ices/ices.sh")
  uri = list.hd(default="", lines)
  # create and return request using this URI
  request.create(uri)
end

# Clean filename (no path)
# extension can be specified but is optional (specify as ".mp3", for instance)
def clean_filename(fn, ~ext="", ~allowtrailingdot=false) =
  # the replacement character
  r = "_"
  fn = string.replace(pattern="[\\*/:<>?|\"]", (fun (s) -> r), string.replace(pattern="^\s+|\s+$", (fun (s) -> ""), fn))
  fn = fn ^ ext
  # Trim again because ext might be bad.
  fn = string.replace(pattern="^\s+|\s+$", (fun (s) -> ""), fn)
  # Windows won't have a "." at end (for folder names)
  if allowtrailingdot == false then
    string.replace(pattern="\.$", (fun (s) -> r), fn)
  else
    fn
  end
end

# brute-force ID source generation
# Note: prepend() takes empty() and fail() as valid sources
# but will still complain.
# Note: IDs should also have ReplayGain meta data.
def build_id_source(m, ~path="/home/matthias/Musik/Other/IDs/") =
  fn = path ^ clean_filename(m["artist"] ^ " - ID.flac")
  if m["artist"] != "" and file.exists(fn) then
    log(label="artist_id", "Prepending artist ID: #{fn}")
    # make stereo to prevent errors if it was a mono recording
    audio_to_stereo(single(fn))
  else
    log(label="artist_id", "No artist ID found")
    fail()
  end
end

# Add a skip function to a source
# when it does not have one by default
def add_skip_command(~command,s) =
  # Register the command:
  server.register(
    usage="skip",
    description="Skip the current song in source.",
    command,
    fun(_) -> begin source.skip(s) "Done!" end
  )
end

# get next track from kPlaylist and
# trim silence (below -40 dB) at beginning and end of tracks
s = eat_blank(request.dynamic(id="s", get_next))

# Add a skip command for use via telnet or the nextsong.sh script
add_skip_command(command="skip", s)

# try to prepend a fitting artist ID, if we have one
s = prepend(s, build_id_source())
s = mksafe(s)

# Apply ReplayGain; this seems to work for MP3, Ogg and FLAC,
# WITHOUT writing anything like APE tags to the files.
s = amplify(1.,override="replaygain_track_gain",s)

s = smart_crossfade(s)

# output 256 kbit/s to the test stream
output.icecast(%mp3(bitrate=256),
  host = "<redacted>", port = 8000,
  password = "<redacted>", mount = "test.mp3",
  s)