Replace now_playing XML input with modern replacements

hairmare commented 5 years ago

This issue tracks replacing the XML based input to nowplaying with something modern.

Currently input from audioplayer is delivered to nowplaying using an XML based file format that looks like follows:

<?xml version="1.0"?>
<now_playing playing="1" timestamp="2019-06-22T11:34:04">
  <song timestamp="2019-06-22T11:34:04">
    <title>Small Talk</title>
    <artist>For Esm&#xE9;</artist>
    <album/>
    <genre>Other</genre>
    <kind>MPEG-Audiodatei</kind>
    <track>1</track>
    <numTracks/>
    <year>2012</year>
    <comments/>
    <time>255</time>
    <bitrate>192</bitrate>
    <rating/>
    <disc/>
    <numDiscs/>
    <playCount>1</playCount>
    <compilation/>
    <composer/>
    <grouping/>
    <urlSource/>
    <file/>
    <artworkID/>
  </song>
</now_playing>

This format goes back to the pre-python days of our ticker solution and it was originally defined by a third party proprietary plugin to iTunes, Winamp and the like. This family of plugins has since been opensourced: https://github.com/brandonfuller/nowplayingplugin

The XML format defined by the plugin has stayed with us over the years as we always adapted new systems (like looky and now klangbecken) to send us the information in the format in the given format.

IMO now is as good as any time to start thinking about replacing the input format with something modern. It looks like we will be in a situation again where the modern stuff will be a bit more complicated since it will take into account more sources than before. Right now the following sources are to be considered.

mAirlist as used in the digitalized studios
upstream icecast sources as used when we do field-broadcast events
audio streaming sources with in-band metadata (ie. a tagged opus stream)
audio streaming sources with out-of-band metadata (ie. AES67 streams with metadata that as it seems can be in a serial side-channel in Livewire+)

These replacements should be integrated in a fashion where we don't need to poll the file system for new files like the current solution does.

hairmare commented 2 years ago

Ideas for replacements:

webhook that receives RaBe CloudEvents
event topic/bus for RaBe CloudEvents
lift and shift style conversion of existing xml into a json representation (keep file transfer api)
combine 1 and 2 and use both webhook events and topics/bus

#	S	W	O	T
1	modern	not distributable/HA	adopt RaBe CloudEvents	RaBe CloudEvents are overengineered for our purposes
2	even more modern	non http based service in the mix	adopt RaBe CloudEvents	RaBe CloudEvents are overengineered for our purposes
3	simplez	no real modernisation, won't warrant v3	quickfix if we need to	architecture stagnates
4	most modern?	use_all_the_things.png	adopt RaBe CloudEvents	RaBe CloudEvents are overengineered for our purposes

I think we should do 1. and keep the options of switching to 2. or 4. available. In each of these cases we still need to define the actual payload of the event, so there is still some need to do something along the lines of 3. when it comes to getting rid of XML (we do not want RaBe CloudEvents with XML payloads).

I'm currently leaning on using the nowplaypadgen.dlplus.CONTENT_TYPES derived from MOT/DL+ classes as the primary influence on what we support in the payload of the events.

    "ITEM.ARTIST": {
        "code": 4,
        "category": CATEGORIES[1],
        "id3v1": "ARTIST",
        "id3v2": "TPE1",
    },

JSON payloads would probably use keys based on the key from CONTENT_TYPES like item.artist or description.getData for DESCRIPTOR.GET_DATA (i'm taking the libery to apply some lowercasing and de-snake-casing). Formats like protobuf might pref using the code points.

So to summarise, i'll implement a webhook that receives RaBe CloudEvents to replace the current file transfer method of delivering updates. To get started the request format of these events should look as follows (represented as YAML, wire-format is JSON).

trackStarted Event

specversion : "1.0"
type : "ch.rabe.api.events.track.v1.trackStarted"
# source URI/URL
source : "..."
subject: null
# value is based on source URI/URL but more qualified
id : "{source_uri}#C234-1234-1234"
time : "2018-04-05T17:31:00Z"
datacontenttype : "application/json"
data : 
  # key ITEM.ARTIST in CONTENT_TYPES
  item.artist : "Hairmare and the Band" 
  # key ITEM.TITLE in CONTENT_TYPES
  item.title : "C L O U D E V E N T W A V E"
  # "Length" in ID3v3 Standard
  item.length: 3600

The CloudEvent data field MUST contain item.artist and item.title. It SHOULD contain item.length which nowplaying can use to generate internal trackFinished events in case it misses the event from the player or none gets sent. The data field MAY contain additional fields based on nowplaypadgen.dlplus.CONTENT_TYPES.

trackFinished Event

(this one is paired down to only highlight the diff to trackStarted)

type : "ch.rabe.api.events.track.v1.trackFinished"
# references the corresponding trackStarted events
subject: "{source_uri}#C234-1234-1234"
# not same as original event!
id : "{source_uri}#C345-2345-2345"
# 1 hour later equals what `item.length` told us (would be earlier if we fade out early)
time : "2018-04-05T18:31:00Z"

hairmare commented 11 months ago

re-opening because i'm questioning if the current HTTP receiver is the way to go and and i want to reconsider using kafka as backing event bus for well actually reasons

radiorabe / nowplaying

Replace now_playing XML input with modern replacements #38

trackStarted Event

trackFinished Event