lastgenre plugin: `[SSL: CERTIFICATE_VERIFY_FAILED]`

[discopatrick]

Problem

Running this command in verbose (-vv) mode:

$ beet -vv lastgenre -f danny byrd

(Using the -f flag because: https://github.com/beetbox/beets/issues/2704)

Led to this problem:

user configuration: /Users/patrick/.config/beets/config.yaml
data directory: /Users/patrick/.config/beets
plugin paths:
/Users/patrick/Music/beets/beetsenv/lib/python2.7/site-packages/pylast/__init__.py:51: UserWarning: You are using pylast with Python 2. Pylast will soon be Python 3 only. More info: https://github.com/pylast/pylast/issues/265
  UserWarning,
Sending event: pluginload
library database: /Volumes/2TB Iomega Data/Users/patrick/Music/beets/library/musiclibrary.blb
library directory: /Volumes/2TB Iomega Data/Users/patrick/Music/beets/collection
Sending event: library_opened
lastgenre: last.fm error: NetworkError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:726)
lastgenre: last.fm error: NetworkError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:726)
lastgenre: genre for album Danny Byrd - Rave Digger (None):
Sending event: database_change
Sending event: database_change
Sending event: database_change
lastgenre: last.fm error: NetworkError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:726)
lastgenre: last.fm error: NetworkError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:726)
Sending event: database_change
lastgenre: genre for track 2017-10-22 01:04:04 - Danny Byrd feat. Liquid - Rave Digger - d:01/01--t:08/13 - Sweet Harmony - 6:03 -  - MP3 - 320kbps - eMusic - dj:True (None):
Sending event: write
Sending event: after_write
lastgenre: last.fm error: NetworkError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:726)
Sending event: database_change
lastgenre: genre for track 2017-10-22 01:04:04 - Danny Byrd - Rave Digger - d:01/01--t:10/13 - Planet Earth - 3:41 -  - MP3 - 320kbps - eMusic - dj:True (None):
Sending event: write
Sending event: after_write
lastgenre: last.fm error: NetworkError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:726)

Setup

• OS: OSX 10.10.5
• Python version: 2.7.15
• beets version: 1.4.7
• Turning off plugins made problem go away (yes/no): No, tried disabling all plugins apart from lastgenre, didn't help.

My configuration (output of beet config) is:

/Users/patrick/Music/beets/beetsenv/lib/python2.7/site-packages/pylast/__init__.py:51: UserWarning: You are using pylast with Python 2. Pylast will soon be Python 3 only. More info: https://github.com/pylast/pylast/issues/265
  UserWarning,

format_item: $added - $artist - $album - d:$disc/$disctotal--t:$track/$tracktotal - $title - $length - $genre - $format - $bitrate - $import_source - dj:$dj
alternatives:
    macbookpro:
        directory: /Volumes/live/Music/beets
        formats: alac mp3
        query: import_source:vinyl , import_source:beatport , dj:True
        removable: yes
    plex:
        directory: /Volumes/2TB Iomega Data/Users/patrick/Music/plex
        formats: flac aac alac mp3
        query: ''
convert:
    dest: ~/Music/beets-convert-test
    format: alac
    never_convert_lossy_files: no
    album_art_maxwidth: 0
    auto: no
    threads: 2
    copy_album_art: no
    tmpdir:
    max_bitrate: 500

    paths: {}
    pretend: no
    quiet: no
    formats:
        alac:
            command: ffmpeg -i $source -y -vn -acodec alac $dest
            extension: m4a
        aac:
            command: ffmpeg -i $source -y -vn -acodec aac -aq 1 $dest
            extension: m4a
        opus: ffmpeg -i $source -y -vn -acodec libopus -ab 96k $dest
        mp3: ffmpeg -i $source -y -vn -aq 2 $dest
        flac: ffmpeg -i $source -y -vn -acodec flac $dest
        ogg: ffmpeg -i $source -y -vn -acodec libvorbis -aq 3 $dest
        wma: ffmpeg -i $source -y -vn -acodec wmav2 -vn $dest
    embed: yes
    no_convert: ''
library: /Volumes/2TB Iomega Data/Users/patrick/Music/beets/library/musiclibrary.blb
lastgenre:
    source: track
    whitelist: no
    count: 1
    prefer_specific: no
    force: yes
    min_weight: 10
    auto: yes
    separator: ', '
    fallback:
    canonical: no
types:
    dj: bool

plugins:
- fetchart
- chroma
- discogs
- fromfilename
- convert
- alternatives
- info
- types
- web
- lastgenre
directory: /Volumes/2TB Iomega Data/Users/patrick/Music/beets/collection

import:
    move: yes
    log: /Volumes/2TB Iomega Data/Users/patrick/Music/beets/log/beets.log
acoustid:
    apikey: REDACTED
sort_item: albumartist+ album+ disc+ track+
web:
    cors_supports_credentials: no
    reverse_proxy: no
    host: 127.0.0.1
    include_paths: no
    cors: ''
    port: 8337
discogs:
    tokenfile: discogs_token.json
    user_token: REDACTED
    apikey: REDACTED
    apisecret: REDACTED
    source_weight: 0.5
chroma:
    auto: yes
fetchart:
    auto: yes
    minwidth: 0
    sources:
    - filesystem
    - coverart
    - itunes
    - amazon
    - albumart
    google_engine: 001442825323518660753:hrh5ch1gjzm
    enforce_ratio: no
    cautious: no
    maxwidth: 0
    store_source: no
    google_key: REDACTED
    fanarttv_key: REDACTED
    cover_names:
    - cover
    - front
    - art
    - album
    - folder

[sampsyo]

SSL errors are fun! 🙄 I know they can be super frustrating, but I'm not exactly sure what to suggest here—other than perhaps trying to install Python 3 instead of using the system's built-in Python? Sometimes it's like dark magic trying to get Python's various URL libraries to find the right system-installed certificate store.

[discopatrick]

Well in my case it would be a case of installing Python 2 because I'm still using a plugin that hasn't been converted yet! But I believe I have installed 2.7.15 in place of the system default of 2.7.13...

[discopatrick]

@sampsyo - can you foresee any problems with running beets in 2 venvs, one in python2 for the beets-alternatives plugin, and another in python3 to fix this problem? Would the library be compatible with both?

[sampsyo]

That should be fine.

I don't really know whether switching to another Python will work, FWIW… maybe it would be just as good to try the built-in macOS Python as to try Python 3? It's really just guessing (at least for me) when it comes to certificate shenanigans.

[discopatrick]

Ah, I just fixed it. Went to reinstall 2.7.15 from the installer and noticed this in the pre-install notes:

---

Certificate verification and OpenSSL_[CHANGED in 2.7.15]

This variant of Python 2.7 now includes its own private copy of OpenSSL 1.0.2. Unlike previous releases, the deprecated Apple-supplied OpenSSL libraries are no longer used. This also means that the trust certificates in system and user keychains managed by the Keychain Access application and the security command line utility are no longer used as defaults by the Python ssl module. A sample command script is included in /Applications/Python 2.7 to install a curated bundle of default root certificates from the third-party certifi package (https://pypi.python.org/pypi/certifi). Click on Install Certificates to run it. If you choose to use certifi, you should consider subscribing to the project's email update service to be notified when the certificate bundle is updated.

The bundled pip included with the Python 2.7 installer has its own default certificate store for verifying download connections.

---

So running /Applications/Python 2.7/Install Certificates.command will install the certificates you need.

[sampsyo]

Great news!