search

mopidy / mopidy

Mopidy is an extensible music server written in Python
https://mopidy.com
Apache License 2.0
8.02k stars 685 forks source link

Metadata not recognized for some flac files #1453

Closed SeeSpotRun closed 8 years ago

SeeSpotRun commented 8 years ago

Flac files with metadata Artist: Foo is recognised but artist: Foo or ARTIST: Foo are not. I can obviously modify my files to the "correct" form (Artist) but I'm wondering whether mopidy should be less case-sensitive here.

(Edit: the above was a mis-diagnosis, ie the problem is not related to capitalisation, but affects some flac files and not others; see-below)

Looking through the code I can't pinpoint where the tag mapping happens so wonder whether this is an upstream issue?

jodal commented 8 years ago

Most of the media scanning work is offloaded to GStreamer. We just released Mopidy 2.0 which upgrades GStreamer to a version with years of new bug fixes. Are you still running Mopidy 1.x/GStreamer 0.10, or have you upgraded to Mopidy 2.x/GStreamer 1.x?

If experiencing this on Mopidy 2.x/GStreamer 1.x, a tag scan of an offending file would be interesting. Try playing the file with gst-launch-1.0 with the -t argument to print found tags. Example:

$ gst-launch-1.0 -t playbin uri=file:///home/jodal/Music/Ralph\ Myerz\ and\ the\ Jack\ Herren\ Band/Your\ New\ Best\ Friends/01\ -\ Kill\ the\ Habit.mp3                                                                                                                                                                
Setting pipeline to PAUSED ...
Pipeline is PREROLLING ...
FOUND TAG      : found by element "pulsesink1".
           title: Kill the Habit
          artist: Ralph Myerz and the Jack Herren Band
           album: Your New Best Friends
        datetime: 2004-09-27
    track number: 1
container format: ID3 tag
     track count: 11
     ID3v2 frame: buffer of 39 bytes
                : buffer of 31 bytes
                : buffer of 33 bytes
                : buffer of 63 bytes
                : buffer of 26 bytes
                : buffer of 47 bytes
                : buffer of 44 bytes
 album artist ID: 1ab67c63-535b-4f92-869b-a42d54c5e91d
       artist ID: 1ab67c63-535b-4f92-869b-a42d54c5e91d
        album ID: 94c2456f-9320-41f2-b51e-8ce7ee70ec77
    album artist: Ralph Myerz and the Jack Herren Band
        track ID: 3d98628f-a912-439e-a710-df54abd69a35
 artist sortname: Ralph Myerz and the Jack Herren Band
       publisher: Emperor Norton
...
SeeSpotRun commented 8 years ago

Thanks for the quick response.

On closer inspection it doesn't look like a Capitalisation problem - it's just that the first dozen or so working / non-working that I looked at seemed to follow that pattern. But the examples below both have metadata as "ARTIST"

$ mopidy --version
Mopidy 2.0.0

Non-working file:

$ gst-launch-1.0 -t playbin uri=file:"///mnt/Music/01 - Speak to Me - Breathe.flac"
Setting pipeline to PAUSED ...
Pipeline is PREROLLING ...
FOUND TAG      : found by element "fake-audio-sink".
    track number: 1
     disc number: 0
           title: Speak to Me / Breathe
          artist: Pink Floyd
           album: The Dark Side of the Moon
           genre: Unknown
        datetime: 1973-01-01
        track ID: 96685213-a25c-4678-9a13-abd9ec81cf35
       artist ID: 83d91898-7763-47d7-b03b-b92132375c47
        album ID: 25fbfbb4-b1ee-4448-aadf-ae3bc2e2dd27
 album artist ID: 83d91898-7763-47d7-b03b-b92132375c47
 artist sortname: Pink Floyd
          discid: 700a0f09
musicbrainz-discid: 8jTZJTgjGGtKjwdt8UyAgK3DnbY-
     track count: 9
    album artist: Pink Floyd
     audio codec: Free Lossless Audio Codec (FLAC)

$ metaflac --list "/mnt/Music/01 - Speak to Me - Breathe.flac"
<snip>
METADATA block #1
  type: 4 (VORBIS_COMMENT)
  is last: false
  length: 580
  vendor string: reference libFLAC 1.2.1 20070917
  comments: 16
    comment[0]: TRACKNUMBER=1
    comment[1]: DISCNUMBER=0
    comment[2]: TITLE=Speak to Me / Breathe
    comment[3]: ARTIST=Pink Floyd

Working file:

$ gst-launch-1.0 -t playbin uri=file:"///mnt/Music/01 - Wonderwall.flac"
Setting pipeline to PAUSED ...
Pipeline is PREROLLING ...
FOUND TAG      : found by element "fake-audio-sink".
           title: Wonderwall
          artist: Oasis
           album: (What's the Story) Morning Glory?
        datetime: 1995
         comment: Track 3
    track number: 3
           genre: Britpop
container format: ID3 tag
     track count: 12
     ID3v2 frame: buffer of 120 bytes
        duration: 258706000000
      encoded by: Exact Audio Copy   (Secure mode)
         encoder: FLAC.EXE -6 -V -T "ARTIST=Oasis" -T "TITLE=Wonderwall" -T "ALBUM=(What's the Story) Morning Glory?" -T "DATE=1995" -T "TRACKNUMBER=03" -T "GENRE=Britpop" -T "COMMENT=Track 3" -o

$ metaflac --list "/mnt/Music/01 - Wonderwall.flac"
<snip>
METADATA block #2
  type: 4 (VORBIS_COMMENT)
  is last: false
  length: 217
  vendor string: reference libFLAC 1.2.1 20070917
  comments: 8
    comment[0]: ARTIST=Oasis
    comment[1]: TITLE=Wonderwall
    comment[2]: ALBUM=(What's the Story) Morning Glory?

How do I check/change what verison of GStreamer mopidy is using? Version 0.1 doesn't seem to be able to read the metadata for the "non-working" files:

$ gst-launch-0.1 -t playbin uri=file:"///mnt/Music/01 - Speak to Me - Breathe.flac"
Setting pipeline to PAUSED ...
Pipeline is PREROLLING ...
connect(2) call to /dev/shm/jack-1000/default/jack_0 failed (err=No such file or directory)
attempt to connect to server failed
connect(2) call to /dev/shm/jack-1000/default/jack_0 failed (err=No such file or directory)
attempt to connect to server failed
ERROR: from element /GstPulseSink:audiosink-actual-sink-pulse: Failed to connect: Connection refused
Additional debug info:
pulsesink.c(570): gst_pulseringbuffer_open_device (): /GstPulseSink:audiosink-actual-sink-pulse
ERROR: pipeline doesn't want to preroll.
Setting pipeline to NULL ...
Freeing pipeline ...
kingosticks commented 8 years ago

Since you're using Mopidy 2.0.0 you are using gstreamer1. 0 and it seems to find all your tags quite happily in both cases. Is this problem with the local or the file backend? If it's with the local backend, did you originally do a scan with a version of Mopidy older than 2.0.0 or is this a full fresh scan?

SeeSpotRun commented 8 years ago

It's the local backend. It's a full fresh scan.

I suspect something is broken in the archlinuxarm package because pacman is showing gstreamer0.10 as a dependency:

# pacman -Si mopidy
Repository      : community
Name            : mopidy
Version         : 2.0.0-1
Description     : Mopidy is an extensible music server written in Python
Architecture    : any
URL             : http://www.mopidy.com
Licenses        : APACHE
Groups          : None
Provides        : None
Depends On      : python2  python2-pykka>=1.1  python2-setuptools
                  python2-requests  gstreamer0.10-python
                  gstreamer0.10-base-plugins  gstreamer0.10-good-plugins
                  gstreamer0.10-ugly-plugins  python2-tornado>=2.3
Optional Deps   : mopidy-spotify: Spotify backend
                  mopidy-scrobbler: Last.FM scrobbler
                  mopidy-mopify: Web client with Spotify-inspired interface
                  mopidy-soundcloud: Soundcloud backend
Conflicts With  : None
Replaces        : None
Download Size   : 2.83 MiB
Installed Size  : 9.20 MiB
Packager        : Arch Linux ARM Build System <builder+xu3@archlinuxarm.org>
Build Date      : Tue Feb 16 18:01:32 2016
Validated By    : MD5 Sum  SHA-256 Sum  Signature
SeeSpotRun commented 8 years ago

Ok I've uninstalled gstreamer0.10 and installed mopidy from source as per https://docs.mopidy.com/en/latest/installation/source/ but am still getting the same issue.

To force a database rebuild is it as simple as rm ~/.local/share/mopidy/local-sqlite/library.db?

Edit: I'm going to try a clean reinstall of archlinuxarm and then have another go.

jodal commented 8 years ago

The maintainer of Mopidy in Arch forgot to update the GStreamer dependency when updating the package to 2.0.0. It was fixed earlier today in the 2.0.0-2 package.

To clear the local library, run mopidy local clear. If you're running mopidy as system service, instead run sudo mopidyctl local clear.

SeeSpotRun commented 8 years ago

Thanks again; still no joy.

Probably not relevant but on the first run of mopidy after a clean install I received the warning: sys:1: PyGIWarning: Gst was imported without specifying a version first. Use gi.require_version('Gst', '1.0') before import to ensure that the right version gets loaded. Edit: this warning was on the very first line of output, before 'INFO Starting Mopidy 2.0.0` However that message does not appear on subsequent runs of mopidy.

I found some downloadable archive.org tracks that give the same issue, eg https://archive.org/download/cj2009-10-05.ku100_at37/cj2009-10-05d01t03.ku100_at37.flac. The metadata is recognised by metaflac and gst-launch-1.0 but not by mopidy local scanner.

The .ogg and .mp3 downloads of the same track from https://archive.org/details/cj2009-10-05.ku100_at37 work fine.

SeeSpotRun commented 8 years ago

For what it's worth, with a vanilla musicbox install, all tags appear to be recognised ok.

$ uname -a
Linux MusicBox 3.18.7-v7+ #755 SMP PREEMPT Thu Feb 12 17:20:48 GMT 2015 armv7l GNU/Linux
$ mopidy --version
Mopidy 0.19.5
SeeSpotRun commented 8 years ago

Problem is repeatable on Ubuntu 15.10 (mopidy 2.0.0 installed via pip). Will try to find time tonight to do some debugging. I got as far as identifying that result.tags comes back empty from https://github.com/mopidy/mopidy/blob/31c894030d01a47255f727f79a3f0735df025f4c/mopidy/local/commands.py#L142.

adamcik commented 8 years ago

What does python mopidy/audio/scan.py "/mnt/Music/01 - Speak to Me - Breathe.flac" say, just for comparison? (You probably have to change the command based on where ever things are installed)

SeeSpotRun commented 8 years ago

@adamcik:

$ python2 /usr/lib/python2.7/site-packages/mopidy/audio/scan.py "/mnt/Music/01 - Speak to Me - Breathe.flac"
uri                    file:///mnt/Music/01%20-%20Speak%20to%20Me%20-%20Breathe.flac
mime                   None
duration               237800
playable               True
seekable               True
tags

The problem seems to be upstream, either in gstreamer or its python bindings.

SeeSpotRun commented 8 years ago

You should be able to reproduce yourself via: $ python audio/scan.py https://archive.org/download/cj2009-10-05.ku100_at37/cj2009-10-05d01t03.ku100_at37.flac Compare against: $ python audio/scan.py https://archive.org/download/cj2009-10-05.ku100_at37/cj2009-10-05d01t03.ku100_at37.ogg

adamcik commented 8 years ago

Hmm, works for me. I also checked which GStreamer element is being used. So either there is something wrong with your GStreamer install or you are timing out. Does this only happen with remote files or also with local ones?

Does doubling or say tripling the timeout value in https://github.com/mopidy/mopidy/blob/develop/mopidy/audio/scan.py#L233 help?

python mopidy/audio/scan.py https://archive.org/download/cj2009-10-05.ku100_at37/cj2009-10-05d01t03.ku100_at37.flac
uri                    https://archive.org/download/cj2009-10-05.ku100_at37/cj2009-10-05d01t03.ku100_at37.flac
mime                   audio/x-flac
duration               13314
playable               True
seekable               True
tags
album                  [u'Live at The Ark, 2009-10-05']
comment                [u'Recorded & Engineered by Mark A. Jay, Immersifi (www.immersifi.com). Source: Neumann Type KU-100. Audio Technica AT37 (quasi-ORTF). Edirol R-44. CuBase LE4']
track-number           [3]
artist                 [u'Cowboy Junkies']
title                  [u'Interlude I [Engineered For Headphone Use]']
datetime               [u'2009']
album-disc-number      [1]
composer               [u'N/A']
genre                  [u'Alternative']
 gst-inspect-1.0 flacdec
Factory Details:
  Rank                     primary (256)
  Long-name                FLAC audio decoder
  Klass                    Codec/Decoder/Audio
  Description              Decodes FLAC lossless audio streams
  Author                   Tim-Philipp Müller <tim@centricular.net>, Wim Taymans <wim.taymans@gmail.com>

Plugin Details:
  Name                     flac
  Description              The FLAC Lossless compressor Codec
  Filename                 /usr/lib/x86_64-linux-gnu/gstreamer-1.0/libgstflac.so
  Version                  1.2.4
  License                  LGPL
  Source module            gst-plugins-good
  Source release date      2014-04-18
  Binary package           GStreamer Good Plugins (Ubuntu)
  Origin URL               https://launchpad.net/distros/ubuntu/+source/gst-plugins-good1.0
SeeSpotRun commented 8 years ago

I have a newer gstreamer element. So maybe it's a gstreamer regression.

$  python audio/scan.py https://archive.org/download/cj2009-10-05.ku100_at37/cj2009-10-05d01t03.ku100_at37.flac
uri                    https://archive.org/download/cj2009-10-05.ku100_at37/cj2009-10-05d01t03.ku100_at37.flac
mime                   None
duration               13314
playable               True
seekable               True
tags

$ gst-inspect-1.0 flacdec
Factory Details:
  Rank                     primary (256)
  Long-name                FLAC audio decoder
  Klass                    Codec/Decoder/Audio
  Description              Decodes FLAC lossless audio streams
  Author                   Tim-Philipp Müller <tim@centricular.net>, Wim Taymans <wim.taymans@gmail.com>

Plugin Details:
  Name                     flac
  Description              The FLAC Lossless compressor Codec
  Filename                 /usr/lib/x86_64-linux-gnu/gstreamer-1.0/libgstflac.so
  Version                  1.6.0
  License                  LGPL
  Source module            gst-plugins-good
  Source release date      2015-09-25
  Binary package           GStreamer Good Plugins (Ubuntu)
  Origin URL               https://launchpad.net/distros/ubuntu/+source/gst-plugins-good1.0

$ uname -a
Linux SeeSpotRun-Ubuntu15 4.2.0-27-generic #32-Ubuntu SMP Fri Jan 22 04:49:08 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
SeeSpotRun commented 8 years ago

Ok I pinned this down to this gstreamer commit: https://cgit.freedesktop.org/gstreamer/gst-plugins-good/commit/?id=a042a9815967b0a6c0dfb65f33d78e02ee1ffb43

Reverting the last part of that commit (ie adding back in:

  /* Push tags */
  if (flacparse->tags) {
    gst_pad_push_event (GST_BASE_PARSE_SRC_PAD (flacparse),
        gst_event_new_tag (flacparse->tags));
    flacparse->tags = NULL;
  }

) fixes the issue.

But given that gst-launch-1.0 can still read the tag data without the above revert, maybe we can find a workaround related to how mopidy interacts with gstreamer...

SeeSpotRun commented 8 years ago

Ok hopefully https://github.com/SeeSpotRun/mopidy/commit/ec52aeb558fb50a7ed2022e73584b8bc1410ce75 fixes this without needing any upstream changes to gstreamer.

SeeSpotRun commented 8 years ago

An update for anyone else seeing this issue; it has been patched in gst-plugins-good so you can either compile that from source, or wait until your distro updates to gst-plugins-good 1.6.4 or 1.7.91, or apply the workaround referenced above.

jodal commented 8 years ago

Fixed by PR #1487.