musicbrainz host, https and ratelimit config options are ignored

[PotcFdk]

Problem

While trying to make the mbsync plugin work with a selfhosted server, I noticed that the ratelimiting was not respected. Further looking into the issue, I noticed that none of the requests were appearing in my server logs. Not only that, but when watching the logs while doing a normal import on a fresh installation without any plugins, nothing appears either.

Edit for clarification: Beets continues to work, but sends requests to musicbrainz.org as if I didn't edit the config.

Setup

• OS: multiple tried, behavior was consistent between NixOS and Arch Linux
• Python version: tried 3.10.5 and 3.10.8 respectively
• beets version: 1.6.0
• Turning off plugins made problem go away (yes/no): no

My configuration (output of beet config) is:

# beet -c poc.yml config

musicbrainz:
    host: mbserver.example.org
    https: yes
    ratelimit: 100

[sampsyo]

Huh; that's pretty strange. Two questions seem relevant here:

• Any chance you could include a verbose log from a run where mbsync uses the wrong server?
• Do you know whether beet import is respecting your configuration correctly (meaning the problem is localized to the mbsync plugin)?

[PotcFdk]

Hi @sampsyo,

Do you know whether beet import is respecting your configuration correctly (meaning the problem is localized to the mbsync plugin)?

Yes, I do know that: it is also not respecting the configuration.

Any chance you could include a verbose log from a run where mbsync uses the wrong server?

Even better, I wrote this proof of concept Dockerfile:

FROM archlinux:latest

# install beets
RUN pacman -Sy --noconfirm && pacman -S beets --noconfirm

# add a sound file for the test
ADD https://github.com/beetbox/beets/raw/master/test/rsrc/bpm.mp3 /example.mp3

# create PoC config file
RUN echo -e "musicbrainz:\n" \
        "    host: mbserver.example.org\n" \
        "    https: yes\n" \
        "    ratelimit: 100\n" > poc.yml

# create PoC script
RUN echo -e "#!/bin/bash\n" \
        "echo \"=== BEETS VERSION:\\n\"\nbeet --version\n" \
        "echo \"\\n=== BEETS CONFIGURATION:\"\nbeet -c poc.yml config\n" \
        "echo \"\\n=== BEETS IMPORT (verbose):\"\nbeet -v -c poc.yml imp example.mp3\n" \
         > entrypoint.sh && chmod +x entrypoint.sh

ENTRYPOINT ./entrypoint.sh

You can run it with e.g. docker build -t mb-poc . && docker run -it --rm mb-poc. Note how I set the host to literally mbserver.example.org (an invalid server) and yet it can communicate with musicbrainz.

If I run the example above, I get an output that looks like this:

=== BEETS VERSION:

beets version 1.6.0
Python version 3.10.8
no plugins loaded

=== BEETS CONFIGURATION:

musicbrainz:
    host: mbserver.example.org
    https: yes
    ratelimit: 100

=== BEETS IMPORT (verbose):
overlaying configuration: poc.yml
no user configuration found at /root/.config/beets/config.yaml
data directory: /root/.config/beets
plugin paths:
Sending event: pluginload
library database: /root/.config/beets/library.db
library directory: /root/Music
Sending event: library_opened
Sending event: import_begin
state file could not be read: [Errno 2] No such file or directory: '/root/.config/beets/state.pickle'
Sending event: import_task_created
Sending event: import_task_start
Looking up: /example.mp3
Tagging the album artist - the album
No album ID found.
Search terms: the album artist - the album
Album might be VA: True
Searching for MusicBrainz releases with: {'release': 'the album', 'artist': 'the album artist', 'tracks': '1'}
Requesting MusicBrainz release 37cf17ed-6b5e-4331-8137-5e32ce7706dd
Sending event: mb_track_extract
Sending event: mb_track_extract
Sending event: mb_track_extract
Sending event: mb_track_extract
Sending event: mb_track_extract
Sending event: mb_track_extract
Sending event: mb_track_extract
Sending event: mb_track_extract
Sending event: mb_album_extract
Sending event: albuminfo_received
Candidate: Album - The Album (37cf17ed-6b5e-4331-8137-5e32ce7706dd)
[...]

[stale[bot]]

Is this still relevant? If so, what is blocking it? Is there anything you can do to help move it forward?

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[PotcFdk]

Is this still relevant?

Yes

[stale[bot]]

Is this still relevant? If so, what is blocking it? Is there anything you can do to help move it forward?

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[sampsyo]

Thanks for the Dockerfile that illustrated what was going on! The issue appears to be a specific interaction between the MusicBrainz settings and the -c "overlay" config option. You can see that the setting does work if you put it in your user config file by adding these lines to the PoC Dockerfile:

RUN mkdir -p /root/.config/beets
RUN cp poc.yml /root/.config/beets/config.yaml

The problem is that these MB settings need some special handling because of the way the MB library needs to be configured at the module level, and the -c overlay thing requires some very special handling so it happens early on, and we are not getting the ordering correct between these two special cases.