!tags.m3u support

[romh-acking]

The issue

Tagging is a long standing issue that has plagued old sequenced rips for over the last 20 years. Typically rips for NSF, GBS, etc. are shared in standalone files. You are forced cycle through the subtunes to find that one song you want to listen to. Additionally, they have a default time like 5 minutes or 3 minutes. There's a few formats like NSFE that address the time issue (but vary in granularity), but each format requires a custom tool, a new format (some of which die an unceremonious death like GBSX), and still don't tackle the issue of subtune cycling.

One solution to this problem was M3U tagging. A fine solution until you see the actual contents of the M3U files: CGB-BHOD-GER.m3u:

# @TITLE       Das Geheimnis der Happy Hippo-Insel
# @ARTIST      Kritzelkratz 3000, Infogrames
# @COMPOSER    Stello Doussis
# @DATE        2000-05-17
# @RIPPER      DevEd
# @TAGGER      DevEd

CGB-BHOD-GER.gbs::GBS,0,Title Screen,2:03,,10
CGB-BHOD-GER.gbs::GBS,1,Der Sonnenstrand,4:43,,10
CGB-BHOD-GER.gbs::GBS,2,Durch den Dunkel-Dschungel,2:51,,10
CGB-BHOD-GER.gbs::GBS,3,In der Lauschingen Lagune,5:40,,10
CGB-BHOD-GER.gbs::GBS,4,Im Bröselbach,5:20,,10
CGB-BHOD-GER.gbs::GBS,5,Auf den Wirren Wetterwarte,7:35,,10

01 Title Screen.m3u:

CGB-BHOD-GER.gbs::GBS,0,Title Screen - Stello Doussis - Das Geheimnis der Happy Hippo-Insel - ©2000 Kritzelkratz 3000\, Infogrames,2:03,,10

This is very close to a perfect solution with a major problem. Each song has metadata shoved into one line to conform to winamp's (and I would assume other's) limitations. When a song it played in winamp from a M3U file, it is only aware of a single line from the M3U file. It's not aware of the comments. Additionally, the individual M3U files for songs are not aware of the metadata in the playlist M3U file so the metadata must be duplicated. Hence when plugin author's resorted to the format you see above. It's a major tech debt item that authors have simply worked around. But in the end, it does the job. However, taggers pay the price because the format is not intuitive.

!tags.m3u

!tags.m3u is a format from the vgmstream libraries, the definitive streamed music library. There's a plethora of streamed formats so most stream rips resort to !tags.m3u and .txtp for extra metadata for all streamed formats. !tags.m3u is a static filename that is located at the base of the rip and contains metadata for all the rips in one file. Amusingly, it's like the format in the codeblock above, but it has a few small change that makes a huge difference.

Basically the idea of a !tags.m3u is that there's two types of variable, @ (global) and % (local). Local variables apply to the song below while global variables apply to all songs. This allows you to not repeat information like for composers, but allows you to have a song with unique composer information, if you so choose.

The purpose of txtp files aren't important for this issue, but once a stream file is opened, vgmstream references !tags.m3u for metadata. It knows where to look because it's a constant filename. It's a single source of truth that prevents data duplication!

I recently created a fork of NezPlug to illustrate how a GBS rip would work with !tags.m3u. Please refer to an example !tags.m3u

# @album    Das Geheimnis der Happy Hippo-Insel
# @company  Kritzelkratz 3000, Infogrames
# @artist   Stello Doussis
# @year     2000-05-17
# @ripper   DevEd
# @tagger   DevEd
# @source   CGB-BHOD-GER.gbs

# %title    Title Screen
# %subtune  0
# %length   0:02:03.000
# %fade     0:00:10.000
01 Title Screen.m3u

# %title    Der Sonnenstrand
# %subtune  1
# %length   0:04:43.000
# %fade     0:00:10.000
02 Der Sonnenstrand.m3u

# %title    Durch den Dunkel-Dschungel
# %subtune  2
# %length   0:02:51.000
# %fade     0:00:10.000
03 Durch den Dunkel-Dschungel.m3u

# %title    In der Lauschigen Lagune
# %subtune  3
# %length   0:05:40.000
# %fade     0:00:10.000
04 In der Lauschigen Lagune.m3u

# %title    Im Bröselbach
# %subtune  4
# %length   0:05:20.000
# %fade     0:00:10.000
05 Im Broselbach.m3u

# %title    Auf den Wirren Wetterwarte
# %subtune  5
# %length   0:07:35.000
# %fade     0:00:10.000
06 Auf der Wirren Wetterwarte.m3u

And a song M3U file, 01 Title Screen.m3u:

01 Title Screen.m3u.gbs

It's a lot more digestible, to say the least. The only hack that I employ is appending the formats' extension in the song m3u so the player recognizes to play a certain plugin. For my case with my NezPlug winamp fork, when 01 Title Screen.m3u is opened in winamp, the only data passed to the plugin is single line entries in the M3U file. The plugin does not know the filename it is opening. Appending .gbs to the filename tells winamp to use a plugin that can play gbs music.

The 01 Title Screen.m3u.gbs in this example is the M3U's own file name (with the .gbs extension, explained why earlier). When the filename is passed to the plugin, the .gbs extension is ignored and the filename is cross referenced with the !tags.m3u file. If the filename is found within !tags.m3u, the local variables (found above the filename in the !tags.m3u) and global variables are fed into the player.

Proposition

I would like to ask if the devs here, after hearing my explanation, would be interested in this repo supporting !tags.m3u? I must remind you that !tags.m3u is an existing standard, but for streamed formats. This file format, along with txtp, was a solution the renowned vgmstream library uses to tag the many different types of streamed music. Sequenced formats such as NSF, GBS, and HES would greatly benefit from being tagged with this format.

[mywave82]

How should this be implemented into the GME API?

gme_open_data() receives a blob of data from the user of the library - so no file I/O is performed by GME. This allows for instance projects like Open Cubic Player which has a virtual file system (data can come from inside .zip files) to feed the file-data from its VFS.

One possible solutions would be to add a new function that could be called after gme_open_data() like gme_open_data_read_metadata() that has a parameter that is a callback function allowing GME access to the file-system.

[ZoomTen]

GME already has a mechanism to load .m3u files under gme/M3u_Playlist.cpp and supports two different formats:

# Game:      Pictionary
# Artist:    Software Creations
# Copyright: 1990 LJN
# Composer:  Tim Follin
# Engineer:  Stephen Ruddy
# Ripping:   Gil Galad
# Tagging:   wrldwzrd89

Pictionary.nsf::NSF,1,Title Screen,1:34,-,10,
# .. more entries here

# @TITLE     Pictionary
# @ARTIST    Software Creations
# @DATE      1990-07
# @COMPOSER  Tim Follin
# @SEQUENCER Tim Follin
# @ENGINEER  Stephen Ruddy
# @RIPPER    Gil Galad
# @TAGGER    wrldwzrd89

Pictionary.nsf::NSF,1,Title Screen,1:34,-,10,
# .. more entries here

Players such as foo_gep (foobar2000) can already make use of these m3u-loading functions. All one would do is to extend it for !tags.m3u, as I have done here, albeit this is currently a shoddy job of doing so. I have only tested the most basic functionality with the demo player.

As for TXTP, that would be a different matter entirely, and might not be in the scope of libgme—correct me if I'm wrong on that one.