search

jprjr / lua-music-visualizer

MIT License
4 stars 0 forks source link

readme

lua-music-visualizer

This is a program to create videos from music files, using Lua. It's the successor to my mpd-visualizer and keeps much of the same API. It's able to decode audio from several formats (FLAC, MP3, WAVE, and raw PCM), making it suitable for creating single videos offline, or for a never-ending livestream with MPD.

Unlike mpd-visualizer, this only allows piping the raw video into some sub-process, whether it's ffmpeg, ffplay. You can save videos locally using ffmpeg to encode, or just use cat and redirect your standard output to a file if you want to save the raw RGB video (warning: this will be a large file).

There's a CLI interface as well as a GUI interface.

Supported formats:

M3U8 Playlists

You can write comments in M3U8 playlists to override audio metadata, as well as simulate sending messages on the visualizer MPD channel.

Metadata overrides apply to the next, upcoming track, and messages are "sent" just before the next track begins to decode. For example:

# title A Custom Title
# artist A Custom Artist
/path/to/song.mp3 # this MP3's track and artist tags will be ignored and set
                  # to "A Custom Title" and "A Custom Artist", respectively.

# message some message
# "some message" will be sent on the "visualizer channel" as soon as song.mp3
# is done playing / just before the next track and will trigger on "onchange"
# call in your lua script.
# album A custom album
/path/to/song.wav # this WAV's album tag will be ignored and set to
                  # "A custom album"

Usage

lua-music-visualizer \
  --about (shows licensing info and quits) \
  --width=1280 (video width) \
  --height=720 (video height) \
  --fps=30 (video fps) \
  --bars=24 (number of spectrum analyzer bars to compute) \
  --samplerate=48000 (input sample rate, only for raw PCM) \
  --channels=2 (input audio channels, only for raw PCM) \
  --resample=48000 (desired output sample rate, off by default) \
  -joff (disable JIT) \
  -l<modulename> (calls require("modulename") when Lua is initialized) \
  --probe (probes music file for meta info, ignores remaining paramters) \
  /path/to/song.mp3/flac/wave/raw \
  /path/to/lua/script.lua \
  prog args...

MPD

lua-music-visualizer will connect to MPD if the MPD_HOST environment variable is set, otherwise it won't connect.

On Unix/Linux, you can connect via TCP/IP or Unix sockets. On Windows, you can only connect using TCP/IP.

If your MPD instances needs a password, use MPD_HOST=password@hostname or MPD_HOST=password@/path/to/socket

If you need to connect on a different TCP port, set the MPD_PORT environment variable.

Requirements

Note that the optional libraries have different licensing, compiling against them may make the resulting binary non-redistributable.

Installation

Hopefully you can just run make. Look at the Makefile if that doesn't work.

Different decoders can be enabled/disabled with your make command. The available parameters (and their default state) are:

I've also included Docker files for cross-compiling to Windows, Linux, and OSX.

The OSX cross-compiler requires a copy of the macOS SDK. Long-story short, you'll have to build your own local copy of my macOS cross-compiler image before using the Docker images.

I've added easy targets to the Makefile for using Docker:

What happens

When lua-music-visualizer starts up, it will attempt to open your songfile and parse metadata. If the MPD_HOST environment variable is set, it will connect to MPD.

Your Lua script should return either a function, or an object.

return function()
  print('making a video frame')
end

Or:

return {
  onframe = function(self)
    print('making a video frame')
  end
}

Or, if you want to be more object-oriented:

local M = {}
M.__index = M

function M.new()
  local self = {}
  setmetatable(self,M)
  return self
end

function M:onframe()
  print('making a video frame')
end

function M:onload()
  print('performing initialization stuff')
end

function M:onunload()
  print('shutting down')
end

function M:onchange(e)
  print('MPD saw a change, type: ' .. e)
end

function M:onreload()
  print('visualizer requested reload')
end

return M.new()

The only required function is onframe.

onchange is used when connected to MPD, and called whenever a new message comes in, or the player status changes (such as the song changing).

onload is called at app start-up, and onunload is called when quitting.

onreload is used to signal a reload. The program does not reload your main script, it just calls your own onreload function, which you could use to perform your own reloading procedures. On UNIX/Linux, the reload is triggered by sending a USR1 signal to the process.

For reference, here's all the function signatures you can expose:

The Lua environment

Globals

Before any script is called, your Lua folder is added to the package.path variable, meaning you can create submodules within your Lua folder and load them using require.

Within your Lua script, you have a few pre-defined global variables:

The global stream object

The stream table has two keys:

The global image object

The image module can load most images, including GIFs. All images have a 2-stage loading process. Initially, it just probes the image for information like height, width, etc. You can then load the image synchronously or asynchronously. If you're loading images in the onload function (that is, at the very beginning of the program's execution), its safe to load images synchronously. Otherwise, you should load images asynchronously.

Scroll down to "Image Instances" for details on image methods like img:load()

The global font object

The font object can load BDF (bitmap) fonts.

Scroll down to "Font Instances" for details on font methods

The global file object

The file object has methods for common file operations:

The global song object

The song object has metadata on the current song. The only guaranteed key is elapsed. Everything else can be nil.

If you're playing from a local FLAC/MP3/WAVE file, these will be pre-populated. If you're connected to MPD, it may take a frame or two for these to be populated.

Image Instances

An image instance has the following methods and properties

If img:load() fails, either asynchronously or synchronously, then the state key will be set to error

Frame instances

Once the image is loaded, it will contain an array of frames. Additionally, stream.video is an instance of a frame

For convenience, most frame functions can be used on the stream object directly, instead of stream.video, ie, stream:get_pixel(x,y) can be used in place of stream.video:get_pixel(x,y)

Font instances

Loaded fonts have the following properties/methods:

Examples

example: square

Draw a white square in the top-left corner:

return function()
  stream.video:draw_rectangle(1,1,200,200,255,255,255)
end

example: stamp image

Load an image and stamp it over the video

-- register a global "img" to use
-- globals can presist across script reloads

img = img or nil

return {
    onload = function()
      img = image.new('something.jpg')
      img:load(false) -- load immediately
    end,
    onframe = function()
      stream.video:stamp_image(img.frames[1],1,1)
    end
}

example: load a background

-- register a global 'bg' variable
bg = bg or nil

return {
    onload = function()
      bg = image.new('something.jpg',stream.video.width,stream.video.height,stream.video.channels)
      bg:load(false) -- load immediately
      -- image will be resized to fill the video frame
    end,
    onframe = function()
      stream.video:set(bg)
    end
}

example: display song title

-- register a global 'f' to use for a font
f = f or nil

return {
    onload = function()
      f = font.new('some-font.bdf')
    end,
    onframe = function()
      if song.title then
          stream.video:stamp_string(f,song.title,3,1,1)
          -- places the song title at top-left (1,1), with a 3x scale
      end
    end
}

example: draw visualizer bars

-- set a maximum height for the bars, in pixels
local bars_height = 100
return {
    onframe = function()
        -- draws visualizer bars
        -- each bar is 10px wide
        -- bar height is between 0 and 100 (bars height)
        for i=1,stream.audio.spectrum_len,1 do
            stream.video:draw_rectangle((i-1)*20, 680 ,10 + (i-1)*20, 680 - (math.ceil(stream.audio.amps[i] * bars_height)) , 255, 255, 255)
        end

    end
}

example: animate a gif

local frametime = 1000 / stream.video.framerate
-- frametime is how long each frame of video lasts in milliseconds
-- we'll use this to figure out when to advance to the next
-- frame of the gif

-- register a global 'gif' variable
gif = gif or nil

return {
    onload = function()
      gif = image.new('agif.gif')
      gif:load(false) -- load immediately

      -- initialize the gif with the first frame and frametime
      gif.frameno = 1
      gif.nextframe = gif.delays[gif.frameno]
    end,
    onframe = function()
      stream.video:stamp_image(gif.frames[gif.frameno],1,1)
      gif.nextframe = gif.nextframe - frametime
      if gif.nextframe <= 0 then
          -- advance to the next frame
          gif.frameno = gif.frameno + 1
          if gif.frameno > gif.framecount then
              gif.frameno = 1
          end
          gif.nextframe = gif.delays[gif.frameno]
      end
    end
}

example: use stamp_string_adv with a function to generate a rainbow

local vga

local colorcounter = 0
local colorprops = {}

local function cycle_color(i, props)
  if i == 1 then
    -- at the beginning of the string, increase our color counter
    colorcounter = colorcounter + 1
    props = {
      x = 1,
    }
  end
  if colorcounter == 36 then
    -- one cycle is 30 degrees
    -- we move 10 degrees per frame, so 36 frames for a full cycle
    colorcounter = 0
  end

  -- use the color counter offset + i to change per-letter colors
  local r, g, b = image.hsl_to_rgb((colorcounter + (i-1) ) * 10, 50, 50)

  -- also for fun, we make each letter drop down
  return {
    x = props.x,
    y = 50 + i * (vga.height/2),
    font = vga,
    scale = 3,
    r = r,
    g = g,
    b = b,
  }
end

local function onload()
  vga = font.load('demos/fonts/7x14.bdf')
end

local function onframe()
  stream:stamp_string(vga, "Just some text", 3, 1, 1, 255, 255, 255)
  stream:stamp_string_adv("Some more text", cycle_color )
end

return {
  onload = onload,
  onframe = onframe,
}

Output:

output of rainbow demo

example: use stamp_string_adv with a function to do the wave

local vga
local sin = math.sin
local ceil = math.ceil

local sincounter = -1
local default_y = 30
local wiggleprops = {}

local function wiggle_letters(i, props)
  if i == 1 then
    sincounter = sincounter + 1
    props = {
      x = 10,
    }
  end
  if sincounter == (26) then
    sincounter = 0
  end

  return {
    x = props.x,
    y = default_y + ceil( sin((sincounter / 4) + i - 1) * 10),
    font = vga,
    scale = 3,
    r = 255,
    g = 255,
    b = 255,
  }
end

local function onload()
  vga = font.load('demos/fonts/7x14.bdf')
end

local function onframe()
  stream:stamp_string_adv("Do the wave", wiggle_letters )
end

return {
  onload = onload,
  onframe = onframe,
}

Output:

output of wave demo

License

Unless otherwise stated, all files are released under an MIT-style license. Details in LICENSE

Some exceptions:

Known users