Better error message when MusicBrainz is unreachable

[sampsyo]

A user recently was confused by a terrible error message when their DNS was misconfigured:

Error: "caused by: <urlopen error [Errno -2] Name or service not known>" in
release search with query blah blah blah...

It would be great to have this instead print "MusicBrainz not reachable" (maybe with more info in the verbose log).

[smittie]

I suspect that "unreachable" is an inaccurate assessment of the failure. I am currently getting repeated MusicBrainz is unreachable errors while musicbrainz.org is completely reachable. The error message is frustrating because I have no way to diagnose and correct whatever the actual problem is. It is not my network.

[sampsyo]

Huh; that's frustrating. Are there any more details in the verbose log?

[smittie]

Happy to try it and see. Do I need to trigger verbose or is the log stored somewhere? On Dec 23, 2015 12:01, "Adrian Sampson" notifications@github.com wrote:

Huh; that's frustrating. Are there any more details in the verbose log?

— Reply to this email directly or view it on GitHub https://github.com/sampsyo/beets/issues/1190#issuecomment-166980678.

[sampsyo]

Thanks! Verbose mode is enabled with -vv on the command line.

[smittie]

I don't see the same errors in verbose mode. Happy to send you whatever info you like to try and diagnose the issue.

...Smittie

On 12/23/2015 12:11 PM, Adrian Sampson wrote:

Thanks! Verbose mode is enabled with |-vv| on the command line.

— Reply to this email directly or view it on GitHub https://github.com/sampsyo/beets/issues/1190#issuecomment-166982447.

[smittie]

Definitely do not see the same errors with verbose turned on.

[sampsyo]

Hmm; it sounds like they may be happening intermittently. Do the errors reliably happen when verbose mode is off? (That is, is the probability of seeing an error affected by -vv?)

[smittie]

OK. No clue here. I ran "beet import ./" three or five times with the same result; musicbrainz not available. I turned on -vv and it ran without incident. Now, runs fine.

In the times when I was getting the not available errors, musicbrainz.org was reachable via ping and http. Likewise, all other internet related activity showed no slowness or outage.

I still think the error message is communicating the wrong information but I also think the errors were on MusicBrainz end of the chain.

What can I do to help figure it out?

...Smittie

On 12/23/2015 01:00 PM, Adrian Sampson wrote:

Hmm; it sounds like they may be happening intermittently. Do the errors reliably happen when verbose mode is off? (That is, is the probability of seeing an error affected by -vv?)

— Reply to this email directly or view it on GitHub https://github.com/sampsyo/beets/issues/1190#issuecomment-166989193.

[sampsyo]

Wow, weird! There must have been something going on with the MB servers. If you ever do catch it again with verbose mode enabled, please do comment again.

[smittie]

MusicBrainz replied to my tweet. They had server problems that affected queries. https://twitter.com/MusicBrainz/status/679781795722858497

[ironbars]

Hi there, sorry to comment on an old issue, but I'm experiencing this same error with verbose mode enabled. MB is definitely reachable from me right now, but beets can't get there for some reason. Python is raising a NetworkError and says the server is responding with HTTP 503, but again MB is available at the time I'm making the request.

[ironbars]

Sorry, meant to leave the output and backtrace:

user configuration: /home/mpatton/.config/beets/config.yaml
data directory: /home/mpatton/.config/beets
plugin paths: 
Sending event: pluginload
library database: /home/mpatton/.config/beets/data/musiclibrary.blb
library directory: /home/mpatton/Music
Sending event: library_opened
Sending event: import_begin
Sending event: import_task_created
Sending event: import_task_start
Looking up: /home/mpatton/Music
Tagging Nujabes - Metaphorical Music
No album IDs found.
Search terms: Nujabes - Metaphorical Music
Album might be VA: True
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/beets/autotag/mb.py", line 341, in match_album
    limit=config['musicbrainz']['searchlimit'].get(int), **criteria)
  File "/usr/lib/python2.7/site-packages/musicbrainzngs/musicbrainz.py", line 965, in search_releases
    return _do_mb_search('release', query, fields, limit, offset, strict)
  File "/usr/lib/python2.7/site-packages/musicbrainzngs/musicbrainz.py", line 769, in _do_mb_search
    return _do_mb_query(entity, '', [], params)
  File "/usr/lib/python2.7/site-packages/musicbrainzngs/musicbrainz.py", line 711, in _do_mb_query
    return _mb_request(path, 'GET', auth_required, args=args)
  File "/usr/lib/python2.7/site-packages/musicbrainzngs/musicbrainz.py", line 400, in __call__
    return self.fun(*args, **kwargs)
  File "/usr/lib/python2.7/site-packages/musicbrainzngs/musicbrainz.py", line 673, in _mb_request
    resp = _safe_read(opener, req, body)
  File "/usr/lib/python2.7/site-packages/musicbrainzngs/musicbrainz.py", line 520, in _safe_read
    raise NetworkError("retried %i times" % max_retries, last_exc)
NetworkError: retried 8 times, caused by: HTTP Error 503: Service Temporarily Unavailable

Error: MusicBrainz not reachable in release search with query {u'release': u'metaphorical music', u'tracks': u'4', u'artist': u'nujabes'}
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/beets/autotag/mb.py", line 341, in match_album
    limit=config['musicbrainz']['searchlimit'].get(int), **criteria)
  File "/usr/lib/python2.7/site-packages/musicbrainzngs/musicbrainz.py", line 965, in search_releases
    return _do_mb_search('release', query, fields, limit, offset, strict)
  File "/usr/lib/python2.7/site-packages/musicbrainzngs/musicbrainz.py", line 769, in _do_mb_search
    return _do_mb_query(entity, '', [], params)
  File "/usr/lib/python2.7/site-packages/musicbrainzngs/musicbrainz.py", line 711, in _do_mb_query
    return _mb_request(path, 'GET', auth_required, args=args)
  File "/usr/lib/python2.7/site-packages/musicbrainzngs/musicbrainz.py", line 400, in __call__
    return self.fun(*args, **kwargs)
  File "/usr/lib/python2.7/site-packages/musicbrainzngs/musicbrainz.py", line 673, in _mb_request
    resp = _safe_read(opener, req, body)
  File "/usr/lib/python2.7/site-packages/musicbrainzngs/musicbrainz.py", line 520, in _safe_read
    raise NetworkError("retried %i times" % max_retries, last_exc)
NetworkError: retried 8 times, caused by: HTTP Error 503: Service Temporarily Unavailable

Error: MusicBrainz not reachable in release search with query {u'release': u'metaphorical music', u'tracks': u'4', u'arid': u'89ad4ac3-39f7-470e-963a-56509c546377'}
Evaluating 0 candidates.

/home/mpatton/Music (4 items)
Sending event: before_choose_candidate
No matching release found for 4 tracks.

[sampsyo]

Hi! MusicBrainz is having intermittent service outages: https://blog.musicbrainz.org/2016/05/16/state-of-the-onion-metabrainz/

[Freso]

We actually had a complete web service outage most of this (Europe time) night: http://chatlogs.metabrainz.org/brainzbot/metabrainz/msg/3596003/ - so musicbrainz.org/ws/ was inaccessible while musicbrainz.org worked fine.

[ironbars]

Yes indeed, things are working fine now. Thanks for the info!

[theFork]

MusicBrainz is reachable via http, but I get the 503 Error also in verbose mode:

mpd@marvin:/mnt/data/vault/NEW$ beet -vv import Laprous/    
user configuration: /var/lib/mpd/.config/beets/config.yaml
data directory: /var/lib/mpd/.config/beets
Sending event: pluginload
library database: /mnt/data/vault/audio/beets_library.blb
library directory: /mnt/data/vault/audio
Sending event: library_opened
Sending event: import_task_start
Looking up: /mnt/data/vault/NEW/Laprous/The Congregation
Tagging Leprous - The Congregation
No album IDs found.
Search terms: Leprous - The Congregation
Album might be VA: False
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/beets-1.3.9-py2.7.egg/beets/autotag/mb.py", line 381, in album_for_id
    RELEASE_INCLUDES)
  File "/usr/local/lib/python2.7/dist-packages/musicbrainzngs-0.5-py2.7.egg/musicbrainzngs/musicbrainz.py", line 803, in get_release_by_id
    return _do_mb_query("release", id, includes, params)
  File "/usr/local/lib/python2.7/dist-packages/musicbrainzngs-0.5-py2.7.egg/musicbrainzngs/musicbrainz.py", line 669, in _do_mb_query
    return _mb_request(path, 'GET', auth_required, args=args)
  File "/usr/local/lib/python2.7/dist-packages/musicbrainzngs-0.5-py2.7.egg/musicbrainzngs/musicbrainz.py", line 374, in __call__
    return self.fun(*args, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/musicbrainzngs-0.5-py2.7.egg/musicbrainzngs/musicbrainz.py", line 634, in _mb_request
    resp = _safe_read(opener, req, body)
  File "/usr/local/lib/python2.7/dist-packages/musicbrainzngs-0.5-py2.7.egg/musicbrainzngs/musicbrainz.py", line 494, in _safe_read
    raise NetworkError("retried %i times" % max_retries, last_exc)
NetworkError: retried 8 times, caused by: HTTP Error 503: Service Temporarily Unavailable

Error: "retried 8 times, caused by: HTTP Error 503: Service Temporarily Unavailable" in get release by ID with query 'ad5bc9fd-09ba-4071-9e4b-bddb74a1d0e7'
Evaluating 0 candidates.

/mnt/data/vault/NEW/Laprous/The Congregation (11 items)
No matching release found for 11 tracks.
For help, see: http://beets.readthedocs.org/en/latest/faq.html#nomatch
[U]se as-is, as Tracks, Group albums, Skip, Enter search, enter Id, aBort?

Are there any workarounds? Or will I just have to wait?

[sampsyo]

That usually does mean that MusicBrainz is temporarily down, for which there is no workaround beyond running your own mirror. Sometimes it can be helpful to ask around in #musicbrainz on Freenode (IRC) to see if there's something special going on.