Ogg embedded covers support

[Oleksiy-Yakovenko]

Original issue 1083 created by Alexey-Yakovenko on 2014-03-20T20:53:09.000Z:

Ogg containers use a tag called METADATA_BLOCK_PICTURE to store a base64 encoded structure containing an image. For Flac, the albumart plugin reads this tag as cover art, using th Flac library, but for other Ogg codecs it doesn't.

I can create a patch to just have the plugin examine more file types in the same way, but possibly it is better to have decoder plugins supply relevant tag data in some defined fashion? In metadata? As a callback for each plugin?

[Oleksiy-Yakovenko]

Comment #1 originally posted by Alexey-Yakovenko on 2014-03-24T07:43:15.000Z:

i have few ideas how to solve that kind of problem, but haven't decided which one to use yet.

1. use generic library for coping with metadata, instead of putting it into each decoder plugin. this means putting all supported tag types into junklib, and expose to plugins via API (the same way as ID3v2 and APEv2 work now). that way, any plugin can get tagging (read/write) support easily, even if decoder of the format is not present. the problem with this approach -- there are many formats and many tags, this will bloat junklib, and add many new dependencies.
2. implement artwork metadata support in a special way: each file stores artwork as some kind of image file (png/jpg/...) embedded in the file, sometimes base64ed, sometimes [g]zipped, sometimes unsynched (id3v2), etc. we can make a generic data structure for that, and expose it like this:

count = 5 1: id="front cover", offset=100, size=1000, mime=image/png, compression=none, base64=true, id3unsync=false 2: id="rear cover", offset 1200, size=2000, mime=image/jpg, compression=gzip, base64=false, id3unsync=true 3: ... .....

this way, any plugin can get full information about images in a file, and easily load them, that will also work well on android, because we can pass this information to java, and handle image loading/decoding in java, instead of bringing native libs. but of course that's read-only access. writing images into tags is a whole different story. but another API can be made on top of this, to make writing possible.

using the 2nd approach will simplify the artwork plugin a lot.

[Oleksiy-Yakovenko]

Comment #2 originally posted by Alexey-Yakovenko on 2014-03-24T13:05:47.000Z:

Agreed. 1) would be nice but bloated and often behind on new tags. 2) is simpler because each decoder knows about its own tags but needs centralised decisions on an internal Deadbeef format for exchanging information about tags and artwork from non-tag sources, plus maybe work in multiple plugins instead of just junklib.

Adding write support is a huge minefield of endless possibilities, but probably Deadbeef doesn't want to do more than the bare minimum? I am just testing a simple context plugin menu action to embed the existing single piece of cover art into the Ogg container. That needs no extra UI. I also have a function to display any existing (METADATA_BLOCK_PICTURE) tags in a format similar to the one you show. Just dumping 100k of base64 into a normal metadata tag didn't seem very sensible (plus the UI shows very long tags as "multiple values" even when there is only one tag).

[Oleksiy-Yakovenko]

Comment #3 originally posted by Alexey-Yakovenko on 2014-03-24T15:49:36.000Z:

probably Deadbeef doesn't want to do more than the bare minimum?

deadbeef supports tag writing quite well, for certain tag formats. the problems come with certain formats that don't have good opensource solution, and are very complex at the same time and/or lack documentation. most notable examples are MP4 and WMA.

another difficult case is editing tags of multi-subtrack files (cuesheets, ogg chains, mp4 chapters, etc).

of course, i would like deadbeef to be first class tagging solution in the future. not bare minimum. it just takes time to implement tag writing for some things.

the UI shows very long tags as "multiple values" even when there is only one tag

that's a temporary hack to fix GTK crash when trying to edit extremely long lines. it was much-much easier to treat those lines as "multiple values", than to implement special case for them. but i have it on todo list.

[Oleksiy-Yakovenko]

Comment #4 originally posted by Alexey-Yakovenko on 2014-04-29T22:56:02.000Z:

Way out in left field, Ogg supports a variety of image formats as multiplexed streams. This was the original preferred method for mixing images with audio, before the clamour caused the METADATA_BLOCK_PICTURE tag to be defined. Of course, with a defined API to expose images to the artwork plugin, a decoder plugin could support anything anyone wanted to code.

Handling images in an actual Ogg stream would be a useful extension, although they aren't common and most tagging tools can't create them. Along with other stream types such as time-coded text (eg. subtitles), these Ogg stream types are really aimed at a more complex application than a music player.

This sort of feature would need either a way for multiple plugins to decode from the same Ogg file, or some sort of library that Ogg-aware plugins could call to access other stream types. Is it possible to do something like having an Ogg plugin that plugins such as Vorbis can use for features beyond just the Vorbis stream?

Although it is theoretically possible to multiplex more than one audio stream into an Ogg link, it isn't clear exactly what an application should do with such link. They are very rare anyway. A more likely mix, although still rare, would be different audio codecs chained (ie. concatenated) together. Currently only one plugin will get hold of the file and will ignore sub-tracks in different codecs. Simply sending the same file to every plugin that accepts the file extension would get all the sub-tracks inserted but could result in duplicates? For example if ffmpeg and aac both insert the same tracks. Could a hypothetical Ogg plugin recognise multiple different codecs in a file and call the necessary decoders?

[Oleksiy-Yakovenko]

Comment #5 originally posted by Alexey-Yakovenko on 2014-04-30T15:35:38.000Z:

Turns out Ogg chains with mixed content type are not recommended and not supported by most libraries. Deadbeef plugins already support playing Vorbis or Opus from multiplexed Ogg containers. although Flac can't do this. Supporting multiple audio codecs multiplexed in Ogg seems pointless (or extremely extremely advanced, for example mapping streams to different channels), perhaps there isn't any need for a single over-arching Ogg plugin.

A set of Ogg library functions could still be useful though.

[Oleksiy-Yakovenko]

Comment #6 originally posted by Alexey-Yakovenko on 2014-04-30T18:53:50.000Z:

I personally don't see any need for this. Even if someone creates such ogg files, he probably uses them exclusively in custom software, which knows how to handle multiple streams. This problem is not exclusive to Ogg container. E.g. windows media (WMx) and MP4 support the same stuff.

[Oleksiy-Yakovenko]

Comment #7 originally posted by Alexey-Yakovenko on 2014-04-30T18:58:51.000Z:

I assume you mean the multiplexed image streams? That is pretty exotic. I don't know any standard tagging software that uses them.

[Oleksiy-Yakovenko]

Comment #8 originally posted by Alexey-Yakovenko on 2014-04-30T19:06:45.000Z:

no, i mean any streams. including multiple audio codecs in single file. did i misunderstand what you have meant?

[Oleksiy-Yakovenko]

Comment #9 originally posted by Alexey-Yakovenko on 2014-04-30T19:18:07.000Z:

Yes, that's pretty exotic too. Just wild ideas when thinking about an architecture to support embedded images.

[Oleksiy-Yakovenko]

Comment #10 originally posted by Alexey-Yakovenko on 2014-06-15T14:11:03.000Z:

I've modified the Opus plugin to write one embedded image directly into the album art cache directory, currently the first validly parsed image tag. This only happens if the artwork embedded tag option is checked, but otherwise doesn't make any attempt to do anything clever like respecting cache intervals, queuing, or deferring image writes. It relies on the artwork plugin to provide the cache filename and to load the image as necessary.

Opus is a good test case since the opusfile library provides a function that takes a tag and returns image data directly. Flac has something similar, but not Vorbis, although the tag format is identical so the library code could be copied.

This is not intended as a proposed way forward, just a quick and dirty method to get embedded image tags showing up in the GUI. It is just asking for race conditions and the two plugins overwriting eachother's files. I think it would be a better approach to have a defined API for the artwork plugin, in its own time and only if appropriate, to call decoder plugins and see if they have a tag image available.

[Oleksiy-Yakovenko]

Comment #11 originally posted by Alexey-Yakovenko on 2014-06-16T22:18:59.000Z:

Cover art doesn't work for streaming tracks :( I set up my test case, the embedded image was successfully written to the cache, then ... nothing. The artwork plugin rejects any track that isn't a local file.

[Oleksiy-Yakovenko]

Comment #12 originally posted by Alexey-Yakovenko on 2014-06-17T22:51:03.000Z:

Cover art is entirely disabled for non-local files, but there doesn't seem to be any reason for this. Is it from before there was VFS? Is there some type of file that will mess up?

I tested without the check and cover art is happily displayed for streaming files, assuming they have suitable metadata.

[Oleksiy-Yakovenko]

Comment #13 originally posted by Alexey-Yakovenko on 2014-06-18T07:30:23.000Z:

there are many metadata related issues with streaming media. mainly, inability to seek (freely).

[Oleksiy-Yakovenko]

Comment #14 originally posted by Alexey-Yakovenko on 2014-06-18T20:35:21.000Z:

I've created a pull request which removes the total block on artwork requests for non-local track files. In the fetcher thread there are still checks to prevent lookups of local cover art files (obviously) and also embedded tags.

So now for a streaming track, the artwork request gets any cache file, then does online lookups. This appears to work for Opus, Vorbis, AAC, and MP3, and should work for anything that has title and artist metadata.

Supporting embedded tags for non-local files (streams, VFS, etc.) is certainly possible, but probably not with the current method. For example, the metadata for a streaming Vorbis track is only sent once, before the audio, so there is no way for the artwork plugin to get this data. Either the artwork plugin would have to request the data from the decoder, or the decoder would have to place it somewhere accessible.

Supporting VFS access would be possible, in seekable cases at least, for the artwork plugin, but it might be better to involve the decoder plugins here too?

[tmose1106]

Has there been any progress with this 'enhancement'? I have been fiddling for a few hours trying to embed album art in my OGG Vorbis audio files, and I have come to the relization that DeadBeef doesn't load the album artwork in the METADATA_BLOCK_PICTURE tag. Meanwhile, Poweramp on my Android device loads this tag just fine. I love DeadBeef, but this seems kinda ridiculous.

[Oleksiy-Yakovenko]

@tmose1106 This issue is about the Desktop version of DeaDBeeF, which doesn't use the same codebase as Android.

The Android version doesn't support reading embedded album art from any file formats.

It will start supporting the same album art features, as desktop version, in version 2.0.

The new album art plugin, which would be compatible with android, is under development right now.

[tmose1106]

Sorry for any confusion. What I'm saying is that Deadbeef on the desktop, with the current git master, does not load the METADATA_BLOCK_PICTURE tag on OGG Vorbis files. It simply lists it as a metadata tag.

Other audio players are able to read this tag just fine. So there is some functionality missing in DeadBeef's embedded art reader.

[Oleksiy-Yakovenko]

@tmose1106 Yes, this is correct, this issue is open specifically to keep track of this missing feature.

As I mentioned above, right now I'm actively developing a new album art plugin, which will hopefully replace the old one soon (in version 0.8), and I'll make sure it supports more file formats than the old one.

[tmose1106]

Alright, thank you. I think I can make due until then.

[Blurrit]

Has there been any progress on this? As of version 1.8.0, DeaDBeeF still does not load album art for opus or ogg files.

I have found a dirty hack to make it work for opus files, namely to change the file extension to mp3, add album art using EasyTAG, and then change the extension back to opus. But I don't think it's a good solution...

[Oleksiy-Yakovenko]

Unfortunately, I could not fit any of the album art work into 1.8 scope. So no, it's still not done.

[richharp825]

Update: ogg album art now works for ogg files, since version 1.9.4. Album art for opus files still does not work.