search

beetbox / beets

music library manager and MusicBrainz tagger
http://beets.io/
MIT License
12.96k stars 1.82k forks source link

crash after resuming import #5156

Open morphodone opened 8 months ago

morphodone commented 8 months ago

Problem

Trying to resume an import over ssh. My laptop fell asleep and the import stopped.

Running this command in verbose (-vv) mode:

$ beet -vv import /media/plex/music

Led to this problem:

Traceback (most recent call last):
  File "/usr/bin/beet", line 33, in <module>
    sys.exit(load_entry_point('beets==1.6.0', 'console_scripts', 'beet')())
  File "/usr/share/beets/beets/ui/__init__.py", line 1285, in main
    _raw_main(args)
  File "/usr/share/beets/beets/ui/__init__.py", line 1272, in _raw_main
    subcommand.func(lib, suboptions, subargs)
  File "/usr/share/beets/beets/ui/commands.py", line 973, in import_func
    import_files(lib, paths, query)
  File "/usr/share/beets/beets/ui/commands.py", line 943, in import_files
    session.run()
  File "/usr/share/beets/beets/importer.py", line 340, in run
    pl.run_parallel(QUEUE_SIZE)
  File "/usr/share/beets/beets/util/pipeline.py", line 446, in run_parallel
    raise exc_info[1].with_traceback(exc_info[2])
  File "/usr/share/beets/beets/util/pipeline.py", line 311, in run
    out = self.coro.send(msg)
  File "/usr/share/beets/beets/util/pipeline.py", line 193, in coro
    func(*(args + (task,)))
  File "/usr/share/beets/beets/importer.py", line 1376, in lookup_candidates
    task.lookup_candidates()
  File "/usr/share/beets/beets/importer.py", line 660, in lookup_candidates
    autotag.tag_album(self.items, search_ids=self.search_ids)
  File "/usr/share/beets/beets/autotag/match.py", line 461, in tag_album
    for matched_candidate in hooks.album_candidates(items,
  File "/usr/share/beets/beets/plugins.py", line 573, in decorated
    for v in generator(*args, **kwargs):
  File "/usr/share/beets/beets/autotag/hooks.py", line 629, in album_candidates
    yield from plugins.candidates(items, artist, album, va_likely,
  File "/usr/share/beets/beets/plugins.py", line 384, in candidates
    yield from plugin.candidates(items, artist, album, va_likely,
  File "/usr/share/beets/beets/plugins.py", line 732, in candidates
    results = self._search_api(query_type='album', filters=query_filters)
  File "/usr/share/beets/beetsplug/deezer.py", line 222, in _search_api
    response.raise_for_status()
  File "/usr/lib/python3/dist-packages/requests/models.py", line 943, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://api.deezer.com/search/album?q=album%3A%22Now+%28Deluxe%29%22+artist%3A%22Shania+Twain%22

Here's a link to the music files that trigger the bug (if relevant):

Setup

My configuration (output of beet config) is:

directory: /media/plex/music
library: ~/data/musiclibrary.db
import:
    copy: no
    duplicate_action: keep
match:
    strong_rec_thresh: 0.10
plugins: discogs deezer

deezer:
    source_weight: 0.0
Dr-Blank commented 8 months ago

deezer api is throwing an error because of literal "Now" followed by open and close parans.

seems like an edge case to me on deezers backend

amogus07 commented 4 months ago

I'm having the same problem

amogus07 commented 4 months ago

It seems there is work to be done making sure api requests won't get blocked. Question marks (?) also seem to be casing problems, and I wouldn't consider question marks in album titles an edge case. Here's an example:

Album Title: The Fox (What Does the Fox Say?)

Traceback (most recent call last):
  File "/home/konstantink/.local/bin/beet", line 8, in <module>
    sys.exit(main())
             ^^^^^^
  File "/home/konstantink/.local/share/pipx/venvs/beets/lib64/python3.12/site-packages/beets/ui/__init__.py", line 1865, in main
    _raw_main(args)
  File "/home/konstantink/.local/share/pipx/venvs/beets/lib64/python3.12/site-packages/beets/ui/__init__.py", line 1852, in _raw_main
    subcommand.func(lib, suboptions, subargs)
  File "/home/konstantink/.local/share/pipx/venvs/beets/lib64/python3.12/site-packages/beets/ui/commands.py", line 1395, in import_func
    import_files(lib, paths, query)
  File "/home/konstantink/.local/share/pipx/venvs/beets/lib64/python3.12/site-packages/beets/ui/commands.py", line 1326, in import_files
    session.run()
  File "/home/konstantink/.local/share/pipx/venvs/beets/lib64/python3.12/site-packages/beets/importer.py", line 360, in run
    pl.run_parallel(QUEUE_SIZE)
  File "/home/konstantink/.local/share/pipx/venvs/beets/lib64/python3.12/site-packages/beets/util/pipeline.py", line 447, in run_parallel
    raise exc_info[1].with_traceback(exc_info[2])
  File "/home/konstantink/.local/share/pipx/venvs/beets/lib64/python3.12/site-packages/beets/util/pipeline.py", line 312, in run
    out = self.coro.send(msg)
          ^^^^^^^^^^^^^^^^^^^
  File "/home/konstantink/.local/share/pipx/venvs/beets/lib64/python3.12/site-packages/beets/util/pipeline.py", line 195, in coro
    func(*(args + (task,)))
  File "/home/konstantink/.local/share/pipx/venvs/beets/lib64/python3.12/site-packages/beets/importer.py", line 1497, in lookup_candidates
    task.lookup_candidates()
  File "/home/konstantink/.local/share/pipx/venvs/beets/lib64/python3.12/site-packages/beets/importer.py", line 688, in lookup_candidates
    artist, album, prop = autotag.tag_album(
                          ^^^^^^^^^^^^^^^^^^
  File "/home/konstantink/.local/share/pipx/venvs/beets/lib64/python3.12/site-packages/beets/autotag/match.py", line 548, in tag_album
    for matched_candidate in hooks.album_candidates(
  File "/home/konstantink/.local/share/pipx/venvs/beets/lib64/python3.12/site-packages/beets/plugins.py", line 593, in decorated
    for v in generator(*args, **kwargs):
  File "/home/konstantink/.local/share/pipx/venvs/beets/lib64/python3.12/site-packages/beets/autotag/hooks.py", line 759, in album_candidates
    yield from plugins.candidates(items, artist, album, va_likely, extra_tags)
  File "/home/konstantink/.local/share/pipx/venvs/beets/lib64/python3.12/site-packages/beets/plugins.py", line 390, in candidates
    yield from plugin.candidates(
               ^^^^^^^^^^^^^^^^^^
  File "/home/konstantink/.local/share/pipx/venvs/beets/lib64/python3.12/site-packages/beets/plugins.py", line 772, in candidates
    results = self._search_api(query_type="album", filters=query_filters)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/konstantink/.local/share/pipx/venvs/beets/lib64/python3.12/site-packages/beetsplug/deezer.py", line 282, in _search_api
    response.raise_for_status()
  File "/home/konstantink/.local/share/pipx/venvs/beets/lib64/python3.12/site-packages/requests/models.py", line 1024, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://api.deezer.com/search/album?q=album%3A%22The+Fox+%28What+Does+the+Fox+Say%3F%29%22+artist%3A%22Ylvis%22
amogus07 commented 2 months ago

I don‘t like the way it‘s handled now. Currently, when there’s a question mark in the title of an album I‘m importing and the crash happens, I manually remove the question mark and run import once again. I think any characters or patterns we know the Deezer API doesn‘t like should be identified and removed at some point before sending the request or something. IDK if I could implement that though…

snejus commented 2 months ago

Ah this makes sense, sorry I misunderstood the actual issue here!

relink2013 commented 1 month ago

I am seeing this exact same issue from an album with a ? in its name. I tried removing the ? from the folder for the album and the problem is still there. Is there something else I need to do to get around this?

The Deezer URL: https://api.deezer.com/search/albumq=album%3A%22What+the+Fuck+is+Wrong+With+You+People%3F%22+artist%3A%22Combichrist%22

Log Output when it gets stuck: (Sorry if the language is offensive, but thats the album it chose to get stuck on.)

Looking up: /downloads/Combichrist copy/What the Fuck is Wrong With You People
Tagging Combichrist - What the Fuck is Wrong With You People?
No album ID found.Search terms: Combichrist - What the Fuck is Wrong With You People?
Album might be VA: Falsedeezer: Searching Deezer for 'album:"What the Fuck is Wrong With You People?" artist:"Combichrist"'
lyrics: fetched lyrics: Combichrist - We Love You - We Rule the World, Motherfuckers
Sending event: database_change
lyrics: lyrics not found: Combichrist - We Love You - Retreat Hell, Part 1
lyrics: fetched lyrics: Combichrist - We Love You - Retreat Hell, Part 2
Sending event: database_change
Sending event: database_change
Sending event: database_change
Sending event: database_change
Sending event: database_change
Sending event: database_change
Sending event: database_change
Sending event: database_change
Sending event: database_change
Sending event: database_change
Sending event: database_change
Sending event: database_change
Sending event: database_change
Sending event: database_change
Sending event: database_change
Sending event: database_change
Sending event: database_change
Sending event: database_change
Sending event: database_change
Sending event: database_change
Sending event: database_change
Sending event: database_change
Sending event: database_change
Sending event: database_change
Sending event: database_change
Sending event: database_change
Sending event: database_change
Traceback (most recent call last):  
  File "/lsiopy/bin/beet", line 8, in <module>    
    sys.exit(main())             
             ^^^^^^  
  File "/lsiopy/lib/python3.12/site-packages/beets/ui/__init__.py", line 1865, in main    
    _raw_main(args)  File "/lsiopy/lib/python3.12/site-packages/beets/ui/__init__.py", line 1852, in _raw_main    
    subcommand.func(lib, suboptions, subargs)  
  File "/lsiopy/lib/python3.12/site-packages/beets/ui/commands.py", line 1395, in import_func    
    import_files(lib, paths, query)  
  File "/lsiopy/lib/python3.12/site-packages/beets/ui/commands.py", line 1326, in import_files    
    session.run()  
  File "/lsiopy/lib/python3.12/site-packages/beets/importer.py", line 360, in run    
    pl.run_parallel(QUEUE_SIZE)  
  File "/lsiopy/lib/python3.12/site-packages/beets/util/pipeline.py", line 447, in run_parallel    
    raise exc_info[1].with_traceback(exc_info[2])  
  File "/lsiopy/lib/python3.12/site-packages/beets/util/pipeline.py", line 312, in run    
    out = self.coro.send(msg)          
          ^^^^^^^^^^^^^^^^^^^  
  File "/lsiopy/lib/python3.12/site-packages/beets/util/pipeline.py", line 195, in coro    
    func(*(args + (task,)))  
  File "/lsiopy/lib/python3.12/site-packages/beets/importer.py", line 1497, in lookup_candidates    
    task.lookup_candidates()  
  File "/lsiopy/lib/python3.12/site-packages/beets/importer.py", line 688, in lookup_candidates    
    artist, album, prop = autotag.tag_album(                          
                          ^^^^^^^^^^^^^^^^^^  
  File "/lsiopy/lib/python3.12/site-packages/beets/autotag/match.py", line 548, in tag_album    
    for matched_candidate in hooks.album_candidates(                             
                             ^^^^^^^^^^^^^^^^^^^^^^^  
  File "/lsiopy/lib/python3.12/site-packages/beets/plugins.py", line 593, in decorated    
    for v in generator(*args, **kwargs):             
             ^^^^^^^^^^^^^^^^^^^^^^^^^^  
  File "/lsiopy/lib/python3.12/site-packages/beets/autotag/hooks.py", line 759, in album_candidates    
    yield from plugins.candidates(items, artist, album, va_likely, extra_tags)  
  File "/lsiopy/lib/python3.12/site-packages/beets/plugins.py", line 390, in candidates    
    yield from plugin.candidates(               
               ^^^^^^^^^^^^^^^^^^  
  File "/lsiopy/lib/python3.12/site-packages/beets/plugins.py", line 772, in candidates    
    results = self._search_api(query_type="album", filters=query_filters)              
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^  
  File "/lsiopy/lib/python3.12/site-packages/beetsplug/deezer.py", line 282, in _search_api    
    response.raise_for_status()  
  File "/lsiopy/lib/python3.12/site-packages/requests/models.py", line 1024, in raise_for_status    
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://api.deezer.com/search/album?q=album%3A%22What+the+Fuck+is+Wrong+With+You+People%3F%22+artist%3A%22Combichrist%22

Im not sure how to get past this error other than just removing that album from my library for now.

amogus07 commented 1 month ago

@relink2013 remove the question marks from the file tags instead and run beet update if you didn't use beet modify. That should work for now