lyrics: If none are found, it exits with `AttributeError: 'NoneType' object has no attribute 'artist'

doronbehar commented 6 years ago

Problem

I search manually for lyrics of a specific track and It exits with the following traceback:

$ beet -vv lyrics artie shaw carioca
user configuration: /home/doron/.config/beets/config.yaml
data directory: /home/doron/.config/beets
plugin paths:
Sending event: pluginload
inline: adding item field isMultiDisc
library database: /var/lib/beets/db
library directory: /var/lib/mpd/music
Sending event: library_opened
lyrics: failed to fetch: https://www.musixmatch.com/lyrics/Artie-Shaw-and-His-Orchestra/Carioca (404)
lyrics: failed to fetch: https://www.musixmatch.com/lyrics/Artie-Shaw/Carioca (404)
lyrics: failed to fetch: https://www.musixmatch.com/es/letras/Buddy-Rich-Artie-Shaw/The-Carioca (404)
lyrics: failed to fetch: https://www.musixmatch.com/lyrics/Artie-Shaw/Carioca (404)
lyrics: lyrics not found: FLAC % Artie Shaw - Begin the Beguine | CD-04 | 25 Carioca
Traceback (most recent call last):
  File "/usr/bin/beet", line 11, in <module>
    load_entry_point('beets==1.4.6', 'console_scripts', 'beet')()
  File "/usr/lib/python3.6/site-packages/beets/ui/__init__.py", line 1256, in main
    _raw_main(args)
  File "/usr/lib/python3.6/site-packages/beets/ui/__init__.py", line 1243, in _raw_main
    subcommand.func(lib, suboptions, subargs)
  File "/usr/lib/python3.6/site-packages/beetsplug/lyrics.py", line 734, in func
    self.writerest(opts.writerest, None)
  File "/usr/lib/python3.6/site-packages/beetsplug/lyrics.py", line 762, in writerest
    self.artist = item.artist.strip()
AttributeError: 'NoneType' object has no attribute 'artist'

Setup

$ uname -ors
Linux 4.14.15-1-ARCH GNU/Linux

Python version: 3.6.4
beets version: 1.4.6
Turning off plugins made problem go away (yes/no): lyrics plugin of course.

My configuration (output of beet config) is:


lyrics:
    bing_lang_from: []
    google_API_key: REDACTED
    sources: google musixmatch
    auto: yes
    bing_client_secret: REDACTED
    bing_lang_to:
    google_engine_ID: REDACTED
    genius_api_key: REDACTED
    fallback:
    force: no
    local: no

musicbrainz:
    pass: REDACTED
    host: musicbrainz.org
    ratelimit: 1
    ratelimit_interval: 1.0
    searchlimit: 10
    user: doronbehar
acoustid:
    apikey: REDACTED
mpd:
    host: localhost
    port: 6600
    password: REDACTED
    music_directory: /var/lib/mpd/music
    rating: yes
    rating_mix: 0.75
library: /var/lib/beets/db
directory: /var/lib/mpd/music

import:
    write: yes
    copy: yes
    move: no
    link: no
    delete: no
    resume: ask
    incremental: no
    quiet_fallback: skip
    none_rec_action: ask
    timid: no
    log: ~/.local/share/beets/log
    autotag: yes
    quiet: no
    singletons: no
    default_action: apply
    languages: []
    detail: no
    flat: no
    group_albums: no
    pretend: no
    search_ids: []
    duplicate_action: ask

clutter: [Thumbs.DB, .DS_Store]
ignore:
- .*
- '*~'
- System Volume Information
- lost+found
ignore_hidden: yes

replace:
    '[\\/]': '-'
    ^\.: _
    '[\x00-\x1f]': _
    '[<>"\?\*\|]': _
    ':': ' -'
    \s+$: ''
    ^\s+: ''
path_sep_replace: _
asciify_paths: no
art_filename: cover
max_filename_length: 0

plugins:
- absubmit
- acousticbrainz
- inline
- mbcollection
- chroma
- fromfilename
- info
- duplicates
- edit
- fetchart
- ipfs
- lastgenre
- missing
- mbsync
- mpdstats
- mpdupdate
- lyrics
pluginpath: []
threaded: yes
timeout: 5.0
per_disc_numbering: yes
verbose: 0
terminal_encoding:
original_date: no
id3v23: no
va_name: Various Artists

ui:
    terminal_width: 80
    length_diff_thresh: 10.0
    color: yes
    colors:
        text_success: green
        text_warning: yellow
        text_error: red
        text_highlight: red
        text_highlight_minor: lightgray
        action_default: turquoise
        action: blue

format_item: ${format} % ${albumartist} - ${album} | CD-${disc} | ${track} ${title}
format_album: $album
time_format: '%Y-%m-%d %H:%M:%S'
format_raw_length: no

sort_album: albumartist+ album+
sort_item: artist+ album+ disc+ track+
sort_case_insensitive: yes
item_fields:
    isMultiDisc: "u'yes' if\n    (int(disctotal) > 1) and (disctotal != '')\nelse\n    u''\n"

paths:
    default: $album%aunique{album, albumartist}/%if{$isMultiDisc,CD-$disc/}$track $title
    singleton: $artist - Singles/$title
    comp: $album%aunique{album, albumartist}/%if{$isMultiDisc,CD-$disc/}$track $title

statefile: ~/.cache/beets/state.pickle
chrome:
    auto: no
ipfs:
    auto: no
lastgenre:
    force: no
    source: track
    whitelist: yes
    min_weight: 10
    count: 1
    fallback:
    canonical: no
    auto: yes
    separator: ', '
    prefer_specific: no

match:
    strong_rec_thresh: 0.04
    medium_rec_thresh: 0.25
    rec_gap_thresh: 0.25
    max_rec:
        missing_tracks: medium
        unmatched_tracks: medium
    distance_weights:
        source: 2.0
        artist: 3.0
        album: 3.0
        media: 1.0
        mediums: 1.0
        year: 1.0
        country: 0.5
        label: 0.5
        catalognum: 0.5
        albumdisambig: 0.5
        album_id: 5.0
        tracks: 2.0
        missing_tracks: 0.9
        unmatched_tracks: 0.6
        track_title: 3.0
        track_artist: 2.0
        track_index: 1.0
        track_length: 2.0
        track_id: 5.0
    preferred:
        countries: []
        media: []
        original_year: no
    ignored: []
    required: []
    track_length_grace: 10
    track_length_max: 30
absubmit:
    auto: yes
    extractor: ''
include: [passwords.yaml]
edit:
    albumfields: album albumartist
    itemfields: track title artist album
    ignore_fields: id path
duplicates:
    album: no
    checksum: ''
    copy: ''
    count: no
    delete: no
    format: ''
    full: no
    keys: []
    merge: no
    move: ''
    path: no
    tiebreak: {}
    strict: no
    tag: ''
acousticbrainz:
    auto: yes
    force: no
    tags: []
chroma:
    auto: yes
missing:
    count: no
    total: no
    album: no
pathfields: {}
album_fields: {}
fetchart:
    auto: yes
    minwidth: 0
    maxwidth: 0
    enforce_ratio: no
    cautious: no
    cover_names:
    - cover
    - front
    - art
    - album
    - folder
    sources:
    - filesystem
    - coverart
    - itunes
    - amazon
    - albumart
    google_key: REDACTED
    google_engine: 001442825323518660753:hrh5ch1gjzm
    fanarttv_key: REDACTED
    store_source: no
mbcollection:
    auto: no
    collection: ''
    remove: no

sampsyo commented 6 years ago

Thanks! This does look like a problem. However, it looks like it's not directly tied to missing the lyrics for a specific album; instead, this crash is happening at the end of the process, after emitting all the lyrics for the matched set of items.

That call to self.writerest(opts.writerest, None) happens after printing out all the lyrics. There, the item parameter is clearly None. Then, we have this relevant stanza from the writerest method:

if item is None or slug(self.artist) != slug(item.artist):
    if self.rest is not None:
        path = os.path.join(directory, 'artists',
                            slug(self.artist) + u'.rst')
        with open(path, 'wb') as output:
            output.write(self.rest.encode('utf-8'))
        self.rest = None
        if item is None:
            return
    self.artist = item.artist.strip()

Saliently, the final line tries to access item.art even when item is None is true—if self.rest is not None.

@anarcat, can you please take a look?

pestrela commented 4 years ago

following, I have the same issue

doronbehar commented 4 years ago

Note you can use the "subscribe" button at the side bar (desktop) or at the bottom of the page (mobile) instead of commenting and thus notifying everyone else subscribed :)

dorianganessa commented 4 years ago

is there any workaround for this?

Sciencentistguy commented 4 years ago

Is there a fix for this yet?

jimbo27 commented 4 years ago

I have got a similar error:

beet lyrics empress rising lyrics: fetched lyrics: Monolord - Empress Rising - Empress Rising Traceback (most recent call last): File "/usr/bin/beet", line 11, in load_entry_point('beets==1.4.9', 'console_scripts', 'beet')() File "/usr/lib/python3.8/site-packages/beets/ui/init.py", line 1266, in main _raw_main(args) File "/usr/lib/python3.8/site-packages/beets/ui/init.py", line 1253, in _raw_main subcommand.func(lib, suboptions, subargs) File "/usr/lib/python3.8/site-packages/beetsplug/lyrics.py", line 745, in func self.fetch_item_lyrics( File "/usr/lib/python3.8/site-packages/beetsplug/lyrics.py", line 840, in fetch_item_lyrics lyrics = [self.get_lyrics(artist, title) for title in titles] File "/usr/lib/python3.8/site-packages/beetsplug/lyrics.py", line 840, in lyrics = [self.get_lyrics(artist, title) for title in titles] File "/usr/lib/python3.8/site-packages/beetsplug/lyrics.py", line 873, in get_lyrics lyrics = backend.fetch(artist, title) File "/usr/lib/python3.8/site-packages/beetsplug/lyrics.py", line 403, in fetch return self.lyrics_from_song_api_path(song_api_path) File "/usr/lib/python3.8/site-packages/beetsplug/lyrics.py", line 375, in lyrics_from_song_apipath lyrics = html.find("div", class="lyrics").get_text() AttributeError: 'NoneType' object has no attribute 'get_text'

After disabling 'genius' in the sources list in config.yaml lyrics runs through without any error. So I guess in my case it has something to do with 'genius'.

Linkandzelda commented 4 years ago

+1 same error as @jimbo27

Linkandzelda commented 4 years ago

Quick fix in the code just to prevent the error, which does not fix the problem with Genius:

File "/usr/lib/python3.8/site-packages/beetsplug/lyrics.py", line 375

if lyrics is None:
    lyrics = ""
else:
    lyrics = lyrics.get_text()

jansc commented 4 years ago

+1 same error as @jimbo27. Applying the quick fix from @Linkandzelda worked for me.

sampsyo commented 4 years ago

Hi! Would any of you be willing to open a pull request with this fix? Thanks!!

Linkandzelda commented 4 years ago

I would suggest investigating it further since all the fix does is return an empty string and only targets @jimbo27 and my error. If there is an underlying problem with Genius lyrics and lyrics is always None, it will always return an empty string.

pestrela commented 4 years ago

Just a quick comment to say that in the end I've built my own python tool JUST to scrape lyrics, inspired from this beets module.

Supported is Genius, MusixMatch, LyricsWorld, FlashLyrics, LyricsFreak, Decoda, SongLyrics, Lyrics24 and MetroLyrics

also added support for automatically redirect certain titles (ie, when searching for "DAFT PUNK VS MODJO - One More Lady - DJ Stein Mix" , search for "MODJO - Lady" instead

https://github.com/pestrela/music/blob/master/traktor/tools_traktor/cue_scrape_lyrics.py

ybnd commented 4 years ago

I would suggest investigating it further since all the fix does is return an empty string and only targets @jimbo27 and my error. If there is an underlying problem with Genius lyrics and lyrics is always None, it will always return an empty string.

@sampsyo I can look into this

ybnd commented 4 years ago

Quick fix in the code just to prevent the error, which does not fix the problem with Genius:

File "/usr/lib/python3.8/site-packages/beetsplug/lyrics.py", line 375
if lyrics is None:
    lyrics = ""
else:
    lyrics = lyrics.get_text()
The Genius issue was already fixed in https://github.com/beetbox/beets/pull/3554, can't replicate it anymore.

sampsyo commented 4 years ago

Fixed by @ybnd in #3612. :smiley:

phirestalker commented 3 years ago

EDIT: Nevermind I don't think I trust tekstowo for lyrics anyway

beetbox / beets

lyrics: If none are found, it exits with `AttributeError: 'NoneType' object has no attribute 'artist' #2805

Problem

Setup