beet update does not correctly parse albumtype and albumtypes resulting in confusing output that appears to be spurious.

nhorlock commented 1 year ago

Running beet update (including pretend mode) shows confused output that suggests that the albumtype and albumtypes fields are likely to be corrupted.

More specifically it would seem that albumtypes is a string but is being treated like a list of strings. As such the albumtype is being set to albumtypes[0] (a logical change) If we have albumtypes = ["album", "remix"] and albumtype = 'remix' this would correctly set the albumtype to be "album" instead of "remix".

Instead albumtypes is a ';' separated string list of albumtypes. "album; remix" and the update command is trying to set albumtype to albumtypes[0] i.e. 'a'

Problem example

beet -vv update -p artist:collide crushed

Led to this problem:

user configuration: /home/neil/.config/beets/config.yaml
data directory: /home/neil/.config/beets
plugin paths: 
lastgenre: Loading canonicalization tree /home/neil/anaconda3/lib/python3.8/site-packages/beetsplug/lastgenre/genres-tree.yaml
Sending event: pluginload
library database: /hdd/music/musiclibrary.db
library directory: /hdd/music.retagged
Sending event: library_opened
Collide - Vortex - Crushed (5AM Heavenly mix)
  albumtype: remix -> a
  albumtypes: album; remix -> ['a', 'l', 'b', 'u', 'm', ';', ' ', 'r', 'e', 'm', 'i', 'x']
Collide - Vortex - Crushed (Fragment mix)
  albumtype: remix -> a
  albumtypes: album; remix -> ['a', 'l', 'b', 'u', 'm', ';', ' ', 'r', 'e', 'm', 'i', 'x']
Collide - Vortex - Crushed (Out of Control mix)
  albumtype: remix -> a
  albumtypes: album; remix -> ['a', 'l', 'b', 'u', 'm', ';', ' ', 'r', 'e', 'm', 'i', 'x']
Collide - Vortex - Crushed (Now Forgotten mix)
  albumtype: remix -> a
  albumtypes: album; remix -> ['a', 'l', 'b', 'u', 'm', ';', ' ', 'r', 'e', 'm', 'i', 'x']
Collide - Vortex - Crushed (Scored mix)
  albumtype: remix -> a
  albumtypes: album; remix -> ['a', 'l', 'b', 'u', 'm', ';', ' ', 'r', 'e', 'm', 'i', 'x']
Sending event: cli_exit

To put context to that, here is the tagging in place on those tracks.

beet ls -f '$album: $title $albumtype {$albumtypes}' artist:collide crushed
Vortex: Crushed (5AM Heavenly mix) remix {album; remix}
Vortex: Crushed (Fragment mix) remix {album; remix}
Vortex: Crushed (Out of Control mix) remix {album; remix}
Vortex: Crushed (Now Forgotten mix) remix {album; remix}
Vortex: Crushed (Scored mix) remix {album; remix}

It is probably worth noting too that the change does not appear to succeed if -p is not specified. The database_change event is triggered but the data does not change to match the "predicted" output shown and in fact remains the same.

My guess is that the informational output is wrong (it is run in a loop in show_model_changes calling _field_diff which has no special handling. The store method that gets called to do the update is being called with different data. (using a print inside db.py Model::store we get.

>>>>>updating record with UPDATE items SET albumtype=?,albumtypes=?,mtime=? WHERE id=? using ['remix', 'album; remix', 0, 705]

I may try to work out a fix but I'm new to beets and still working out how things hang together.

Setup

OS: Ubuntiu 20.04 LTS
Python version: 3.8.8
beets version: 1.6.0

My configuration (output of beet config) is:

directory: /hdd/music.retagged
library: /hdd/music/musiclibrary.db
duplicate_action: merge

match:
    preferred:
        countries:
        - GB|UK
        - XE
        - XW
        - US
        media: [CD]
    max_rec:
        missing_tracks: none
        unmatched_tracks: medium
        track_id: medium
    distance_weights:
        missing_tracks: 7.1
        track_id: 2.0

paths:
    default: '%lower{$albumartist}/%lower{$album%aunique{}}/%lower{${track}_${title}}'
    singleton: non-album/%lower{$artist}/%lower{$title}
    comp: compilations/%lower{$album%aunique{}}/%lower{$track}_%lower{$title}
    genre:audiobook: audiobooks/%lower{$album%aunique{}}/%lower{$track}_%lower{$title}

replace:
    '[\\/]': _
    ^\.: _
    '[\x00-\x1f]': _
    '[<>:"\?\*\|]': _
    \.$: _
    \s+$: ''
    ^\s+: ''
    ^-: _
    \s+: _
asciify_paths: yes

aunique:
    keys: albumartist album
    disambiguators: albumtype albumdisambig releasegroupdisambig year label catalognum
    bracket: '[]'

plugins: chroma discogs spotify fromfilename duplicates fetchart lastgenre lastimport mbcollection missing web fuzzy
chroma:
    auto: yes
acoustid:
    apikey: REDACTED
discogs:
    user_token: REDACTED
    index_tracks: yes
    apikey: REDACTED
    apisecret: REDACTED
    tokenfile: discogs_token.json
    source_weight: 0.5
    separator: ', '
duplicates:
    tiebreak:
        items: [bitrate]
    album: no
    checksum: ''
    copy: ''
    count: no
    delete: no
    format: ''
    full: no
    keys: []
    merge: no
    move: ''
    path: no
    strict: no
    tag: ''
fetchart:
    default: filesystem coverart itunes amazon albumart lastfm wikipedia
    lastfm_key: REDACTED
    auto: yes
    minwidth: 0
    maxwidth: 0
    quality: 0
    max_filesize: 0
    enforce_ratio: no
    cautious: no
    cover_names:
    - cover
    - front
    - art
    - album
    - folder
    sources:
    - filesystem
    - coverart
    - itunes
    - amazon
    - albumart
    google_key: REDACTED
    google_engine: 001442825323518660753:hrh5ch1gjzm
    fanarttv_key: REDACTED
    store_source: no
    high_resolution: no
    deinterlace: no
    cover_format:
lastgenre:
    canonical: yes
    whitelist: yes
    min_weight: 10
    count: 1
    fallback:
    source: album
    force: yes
    auto: yes
    separator: ', '
    prefer_specific: no
    title_case: yes
lastfm:
    user: nhorlock
    api_key: REDACTED

musicbrainz:
    user: nhorlock
    pass: REDACTED
mbcollection:
    collection: 341616bb-7e1b-4b75-bebb-6f0b382093f0
    auto: yes
    remove: no
spotify:
    source_weight: 0.7
    region_filter: UK
    mode: list
    tiebreak: popularity
    show_failures: no
    artist_field: albumartist
    album_field: album
    track_field: title
    regex: []
    client_id: 4e414367a1d14c75a5c5129a627fcab8
    client_secret: REDACTED
    tokenfile: spotify_token.json
replaygain:
    backend: audiotools
    overwrite: yes
    parallel_on_import: yes
fuzzy:
    prefix: '~'
    threshold: 0.7
missing:
    count: no
    total: no
    album: no
web:
    host: 127.0.0.1
    port: 8337
    cors: ''
    cors_supports_credentials: no
    reverse_proxy: no
    include_paths: no
    readonly: yes
lastimport:
    per_page: 500
    retry_limit: 3

nhorlock commented 1 year ago

Actually, it is worse than I thought. The spurious changes do get applied (though not always, it seems) Here is a example of data after update has been applied beet ls -f '$album: $title $albumtype {$albumtypes}' artist:dead can dance Aion: The Arrival and the Reunion a {['a', 'l', 'b', 'u', 'm']} Aion: Saltarello a {['a', 'l', 'b', 'u', 'm']} Aion: Mephisto a {['a', 'l', 'b', 'u', 'm']} Aion: The Song of the Sibyl a {['a', 'l', 'b', 'u', 'm']} Aion: Fortune Presents Gifts Not According to the Book a {['a', 'l', 'b', 'u', 'm']} Aion: As the Bell Rings the Maypole Spins a {['a', 'l', 'b', 'u', 'm']}

nhorlock commented 1 year ago

Closing this... I just stumbled upon a discussion in 1.6.1 abouthow to manually fix this. Sorry for the duplicate report.

djdembeck commented 1 year ago

For future reference, this is how to fix the problem (once you are updated to 1.6.1): https://github.com/beetbox/beets/pull/4582#issuecomment-1445023493

beetbox / beets