beatport: Fetch musical key, BPM, and genre data

[itsbrex]

Problem

Everything seems to be getting pulled in except BPM & Key. Genre tag appears to be hit or miss. All tracks are from beatport, so the tags should be there. Looking at the API docs I see the facets listed. Are these actually accessible in Track Info?

Requires catalog API Permission 

Content can be filtered on facets. Ex:facets=genreName:House, subGenreName:Hard 

Use returnFacets to obtain specific information about the return set.

Allowed facets/returnFacets:

Note: Not all facets are relevant to all content types.

artistName
performerName
artist
artistId
performerId
label
labelName
labelId
subGenre
subGenreName
subGenreId
genre
genreName
genreId
bpm         <------------
key          <------------
tagId

Setup

• OS: Mac OS X 10.11.5 (15F34)
• Python version: 2,7
• beets version: 1.3.18
• Turning off plugins made problem go away (yes/no): no

My configuration (output of beet config) is:

briansmac:beets anodigital$ beet config
lyrics:
    bing_lang_from: []
    bing_client_secret: REDACTED
    google_API_key: REDACTED
    force: no
    google_engine_ID: REDACTED
    sources:
    - google
    - lyricwiki
    - lyrics.com
    - musixmatch
    auto: yes
    fallback:
    genius_api_key: REDACTED
    bing_lang_to:
library: /Users/anodigital/Desktop/music.blb
original_date: yes
embedart:
    maxwidth: 1000
    compare_threshold: 0
    auto: yes
    ifempty: no
    remove_art_file: no
pluginpath: ~/github/beets/beetsplug

plugins: beatport fromfilename discogs embedart fetchart ftintitle fuzzy importadded lyrics mbsync spotify types
beatport:
    source_weight: 0.5
    tokenfile: beatport_token.json
    apikey: REDACTED
    apisecret: REDACTED
threaded: yes
art_filename: cover
duplicate_action: remove
none_rec_action: skip
ignore: .AppleDouble ._* *~ .DS_Store
directory: /Users/anodigital/Desktop/Music

import:
    copy: no
    write: yes
    resume: yes
    quiet_fallback: skip
    timid: no
    log: beetslog.txt
group_albums: no

match:
    strong_rec_thresh: 0.05
    medium_rec_thresh: 0.25
    rec_gap_thresh: 0.25
    max_rec:
        missing_tracks: medium
        unmatched_tracks: medium
    ignored: missing_tracks unmatched_tracks
discogs:
    source_weight: 0.5
    tokenfile: discogs_token.json
    apikey: REDACTED
    apisecret: REDACTED
fuzzy:
    threshold: 0.7
    prefix: '~'
spotify:
    regex: []
    artist_field: albumartist
    mode: list
    track_field: title
    album_field: album
    region_filter:
    show_failures: no
    tiebreak: popularity
importadded:
    preserve_write_mtimes: no
    preserve_mtimes: no
fetchart:
    auto: yes
    minwidth: 0
    sources:
    - filesystem
    - coverart
    - itunes
    - amazon
    - albumart
    google_engine: 001442825323518660753:hrh5ch1gjzm
    enforce_ratio: no
    cautious: no
    maxwidth: 0
    store_source: no
    google_key: REDACTED
    fanarttv_key: REDACTED
    cover_names:
    - cover
    - front
    - art
    - album
    - folder
ftintitle:
    auto: yes
    drop: no
    format: feat. {0}

[sampsyo]

Yep, it looks like the plugin doesn't get those fields yet. Seems like a reasonable addition if you're interested.

[itsbrex]

@sampsyo definitely interested! 😄

[jbaiter]

I'd love to implement this, but can you tell me how I can pass this information to beets, @sampsyo? I don't see a bpm or key field on the TrackInfo class.

As for the Genre tags, do you have a few IDs of releases that are missing genre tags so I can reproduce this?

[sampsyo]

Ah, good point @jbaiter -- I had forgotten that would be a roadblock.

For purposes like this, I've been meaning to replace the current implementations of TrackInfo and AlbumInfo with an "unstructured dict-like" alternative, where you could add arbitrary fields which will then be transferred into Item and Album objects when they're applied. Maybe this is the excuse we need to implement that style, which would then make this feature easy?

[jbaiter]

Sounds good to me, however I'm afraid my knowledge of beets internals is too limited to tackle that task. If you provide the API, however, I'll gladly adapt the plugin :-)

[sampsyo]

It's a deal—I'll check back in here after I make progress on that refactoring.

[itsbrex]

@sampsyo just wondering if you have made any progress on this. Not a programmer myself, otherwise I'd take a stab at it. Apologize in advance if I sound snobby, not meaning to. 😄

[sampsyo]

No, sorry—I've been distracted with a few other things. I do hope to get to it soon.

[itsbrex]

@sampsyo Right on, no worries. :)

[itsbrex]

I've posted a bounty for this to get implemented if anyone is interested. I just really need this feature so figured I'd put up some cash to make it happen! 😄

https://www.bountysource.com/issues/35434835-beatport-fetch-musical-key-and-bpm-data

[itsbrex]

Btw, I've noticed the Genre tag is missing as well now on every track I import.

[itsbrex]

Anybody up for tackling this? https://www.bountysource.com/issues/35434835-beatport-fetch-musical-key-and-bpm-data

[exislow]

I really need this, too! Is anybody up for it?

[vifino]

I too would like to see this. The BPM/Genre is quite important for me. On Beatport, it's spot-on all the time.

[wualex]

Would also love to see this implemented and added some $ to the bounty :)

https://www.bountysource.com/issues/35434835-beatport-fetch-musical-key-bpm-and-genre-data

@brianroach you seem to have had the same issues as i do now (including the one with "original mix" & not finding some tracks), did you find any other solution after all?

[temrix]

I am currently tackling this issue.

I have already implemented BPM. Musical key should be equivalently easy. Beatport provides genre data for each track while beets has a genre per album. I am currently testing how this can be done best.

However, the API hooks need to be modified for this. Maybe I can modify it such that they take arbitrary values.

[wualex]

Silly question now... is there an "easier" way to update my beets to include this fix than to get the code from Github and install manually? i have it on an Ubuntu 18.04 vm, installed by pip

[temrix]

pip can directly install the code from a git url: pip install git+https://github.com/beetbox/beets.git

This will clone the repo to a temporary directory and then install it via pip.

[wualex]

It seems that it still doesn't get the tags properly from beatport

Just for testing i tried the following tracks by using beet import -s /test/path (tried without -s also): Alex Niggemann - Bwana (afro house) Avidus - Psalm 23 (melodic house & techno) Davi - The Gates Of Babylon (progressive house) Stefano Ritteri - Mutanda (deep house)

...and selected beatport as source on every one but it still doesn't get the genre from it, at least

# metaflac --export-tags-to=- \$albumartist_The\ Gates\ of\ Babylon.flac | grep -i genre GENRE=Electro

The other 3 tracks are tagged as genre Dance Am i doing something wrong or... ?

my config:

paths:
    default: $genre/$year_$month/$albumartist_$title
    singleton: $genre/$year_$month/$albumartist_$title
library: /mnt/d/Music/musiclibrary.db

replace:
    '[\\/]': _
    ^\.: _
    '[\x00-\x1f]': _
    '[<>:"\?\*\|]': _
    \.$: _
    \s+$: ''
original_date: yes
path_sep_replace: _

plugins: inline beatport
directory: /mnt/d/Music

import:
    move: no
    copy: yes
    write: yes
beatport:
    source_weight: 0.0
    tokenfile: beatport_token.json
    apikey: REDACTED
    apisecret: REDACTED
pathfields: {}
album_fields: {}
item_fields: {}

[temrix]

I guess, the data is coming from somewhere else, maybe discogs.

How did you even select the source? I have not found out myself how I can select to use Beatport other than doing `beet import ~/music/ --search-id [id].

[wualex]

It should take it from beatport, otherwise what's the point? :)

By "select the source" i was referring when beets asks me from where to correct the tags on import, you have different candidates options, and some of them would be from beatport, i would select that But after i added source_weight to the config it mostly says it gets the data from beatport by default, otherwise i would manually pick beatport

But now that i look at the similarity line from when i import i see that it shows what tags it found and wants to correct, and from all my tests and options given by beets it only seems to want to correct artist, title, source, length - so it doesn't take into consideration genre at all, it just assumes the default (discogs?) is fine, when... it's not really 😬

@sampsyo could you shed some light on this please?

[sampsyo]

Beets does not use genre for matching. But it will apply new genres to your music, if they're available from the data source. Those won't show up in that list you're referring to, though, which only shows information about the matches.

Also, to see the data on your music, I recommend using something like beet ls -f '$title - $genre' or beet ls -af '$album - $genre'.

@wualex, when you report a problem, it's really useful to provide a complete set of steps so the developer can reproduce the problem. Maybe you can think carefully about how to help @temrix do exactly what you're doing to produce the same results.

[wualex]

At this point the only step and only thing i'm doing is beets import -s /mnt/d/test and even though beets says it corrects tags from beatport as you can see below:

/mnt/d/test/Alex Niggemann - Bwana (Original Mix).flac
Correcting track tags from:
    Alex Niggemann - Bwana (Original Mix) 
To:
    Alex Niggemann - Bwana
URL:
    https://beatport.com/track/bwana-original-mix/11381810
(Similarity: 93.8%) (title) (Beatport)
[A]pply, More candidates, Skip, Use as-is, Enter search, enter Id, aBort? A

/mnt/d/test/Avidus - Psalm 23.flac
Tagging track: Avidus - Psalm 23
URL:
    https://beatport.com/track/psalm-23-original-mix/11717455
(Similarity: 100.0%) (Beatport)

/mnt/d/test/Davi - The Gates Of Babylon (Original Mix).flac
Correcting track tags from:  
  Davi - The Gates Of Babylon (Original Mix)
To: 
   DAVI - The Gates Of Babylon
URL:
   https://beatport.com/track/the-gates-of-babylon-original-mix/6061936
(Similarity: 96.5%) (title) (Beatport)

/mnt/d/test/Kellerkind - Shakti Pan (feat. Sven).flac
Correcting track tags from:
   Kellerkind;Sven - Shakti Pan (feat. Sven)
To: 
   Kellerkind - Shakti Pan feat. Sven
URL:
   https://beatport.com/track/shakti-pan-feat-sven-original-mix/8712463
(Similarity: 94.3%) (artist) (Beatport)
[A]pply, More candidates, Skip, Use as-is, Enter search, enter Id, aBort? A

/mnt/d/test/Tunnelvisions - Guava (Extended Mix).flac
Tagging track: Tunnelvisions - Guava (Extended Mix)
URL: 
   https://beatport.com/track/guava-extended-mix/10227696
(Similarity: 100.0%) (Beatport)

The result is as you can see wrong:

beet ls -f '$artist - $title - $genre'

Alex Niggemann - Bwana - Electro
Avidus - Psalm 23 - Electro
DAVI - The Gates Of Babylon - Electro
Kellerkind - Shakti Pan feat. Sven - Electro
Tunnelvisions - Guava (Extended Mix) - Dance

And now i don't know what else i can provide but i'm up for more testing or if you have questions.... Thanks.

[flovanc]

maybe there is a mistake between "genre" and "subgenre" from beatport. what we need is "subgenre" from beatport copied inside genre id3 from audiofile ... edit: after investigation on beatport api i'm wrong ... i wait for news

[temrix]

I have finally reproduced it and I will file a new issue because this here is already closed. In the new issue I will explain more thoroughly what the problem actually is. Just give me some time.