Python error

[janaxhell]

Here I am again. After more than an hour I took a look at Navidrome and nothing new appeared. So I checked the Subdify log and found this:

2024-09-12 16:59:07 ERROR    Job "artist_reccomendations (trigger: interval[1:00:00], next run at: 2024-09-12 17:59:07 CEST)" raised an exception
Traceback (most recent call last):
  File "/home/user/.local/lib/python3.10/site-packages/apscheduler/executors/base.py", line 125, in run_job
    retval = job.func(*job.args, **job.kwargs)
  File "/home/user/subtify/./main.py", line 145, in artist_reccomendations
    generate_playlists.show_recommendations_for_artist(random.choice(generate_playlists.get_artists_array_names()))
  File "/home/user/subtify/./generate_playlists.py", line 41, in get_artists_array_names
    artists = pysonic.getArtists()
  File "/home/user/.local/lib/python3.10/site-packages/libsonic/connection.py", line 1611, in getArtists
    res = self._doInfoReq(req)
  File "/home/user/.local/lib/python3.10/site-packages/libsonic/connection.py", line 2720, in _doInfoReq
    res = self._opener.open(req)
  File "/usr/local/lib/python3.10/urllib/request.py", line 525, in open
    response = meth(req, response)
  File "/usr/local/lib/python3.10/urllib/request.py", line 634, in http_response
    response = self.parent.error(
  File "/usr/local/lib/python3.10/urllib/request.py", line 563, in error
    return self._call_chain(*args)
  File "/usr/local/lib/python3.10/urllib/request.py", line 496, in _call_chain
    result = func(*args)
  File "/usr/local/lib/python3.10/urllib/request.py", line 643, in http_error_default
    raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 405: Method Not Allowed

Everything looks fine, the service is green, but something still blocks it.

[blastbeng]

Here I am again. After more than an hour I took a look at Navidrome and nothing new appeared. So I checked the Subdify log and found this:

2024-09-12 16:59:07 ERROR    Job "artist_reccomendations (trigger: interval[1:00:00], next run at: 2024-09-12 17:59:07 CEST)" raised an exception
Traceback (most recent call last):
  File "/home/user/.local/lib/python3.10/site-packages/apscheduler/executors/base.py", line 125, in run_job
    retval = job.func(*job.args, **job.kwargs)
  File "/home/user/subtify/./main.py", line 145, in artist_reccomendations
    generate_playlists.show_recommendations_for_artist(random.choice(generate_playlists.get_artists_array_names()))
  File "/home/user/subtify/./generate_playlists.py", line 41, in get_artists_array_names
    artists = pysonic.getArtists()
  File "/home/user/.local/lib/python3.10/site-packages/libsonic/connection.py", line 1611, in getArtists
    res = self._doInfoReq(req)
  File "/home/user/.local/lib/python3.10/site-packages/libsonic/connection.py", line 2720, in _doInfoReq
    res = self._opener.open(req)
  File "/usr/local/lib/python3.10/urllib/request.py", line 525, in open
    response = meth(req, response)
  File "/usr/local/lib/python3.10/urllib/request.py", line 634, in http_response
    response = self.parent.error(
  File "/usr/local/lib/python3.10/urllib/request.py", line 563, in error
    return self._call_chain(*args)
  File "/usr/local/lib/python3.10/urllib/request.py", line 496, in _call_chain
    result = func(*args)
  File "/usr/local/lib/python3.10/urllib/request.py", line 643, in http_error_default
    raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 405: Method Not Allowed

Everything looks fine, the service is green, but something still blocks it.

This is a response from pysonic, subtify can't connect to your navidrome/subsonic server. Check your credentials and your ip/port

Also try to ping the navidrome instance from inside the subtify container.

Or just put the compose file inside the navidrome compose file and add them to the same network.

[janaxhell]

Credentials and ports are ok, I can't ping from inside container (in Portainer), because "ping" is missing and cannot be installed. Anyway, I have merged the 2 composes, will see what happens and check back.

[blastbeng]

Credentials and ports are ok, I can't ping from inside container (in Portainer), because "ping" is missing and cannot be installed. Anyway, I have merged the 2 composes, will see what happens and check back.

I am 100% sure you have some networking issues in your docker setup, I have replicated the same error setting a wrong navidrome ip in my config

This night I will upload a sample yaml using docker networks

[janaxhell]

I don't think it has anything to do with wrong IP, Portainer in the past months had been incompatible with Docker 26 exactly in the Console section and only a couple of weeks ago it's been fixed, but seems not so well. I never had problems with that before, when a component was missing I could simply apt install it and use it. I have 75 containers, all working fine, the only one not working is yours: allow me to doubt I have a networking problem. Now, while Navidrome is still scanning the library after deploying the merged compose, Subtify still gives the same error

2024-09-12 19:33:21 ERROR    Job "artist_reccomendations (trigger: interval[1:00:00], next run at: 2024-09-12 20:33:21 CEST)" raised an exception
Traceback (most recent call last):
  File "/home/user/.local/lib/python3.10/site-packages/apscheduler/executors/base.py", line 125, in run_job
    retval = job.func(*job.args, **job.kwargs)
  File "/home/user/subtify/./main.py", line 145, in artist_reccomendations
    generate_playlists.show_recommendations_for_artist(random.choice(generate_playlists.get_artists_array_names()))
  File "/home/user/subtify/./generate_playlists.py", line 41, in get_artists_array_names
    artists = pysonic.getArtists()
  File "/home/user/.local/lib/python3.10/site-packages/libsonic/connection.py", line 1611, in getArtists
    res = self._doInfoReq(req)
  File "/home/user/.local/lib/python3.10/site-packages/libsonic/connection.py", line 2720, in _doInfoReq
    res = self._opener.open(req)
  File "/usr/local/lib/python3.10/urllib/request.py", line 525, in open
    response = meth(req, response)
  File "/usr/local/lib/python3.10/urllib/request.py", line 634, in http_response
    response = self.parent.error(
  File "/usr/local/lib/python3.10/urllib/request.py", line 563, in error
    return self._call_chain(*args)
  File "/usr/local/lib/python3.10/urllib/request.py", line 496, in _call_chain
    result = func(*args)
  File "/usr/local/lib/python3.10/urllib/request.py", line 643, in http_error_default
    raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 405: Method Not Allowed

Navidrome works fine, Subtify in the same compose does not. And the script says:

root@openmediavault:~/subtify# ./first_run.sh
Traceback (most recent call last):
  File "init.py", line 1, in <module>
    import spotipy
ImportError: No module named spotipy

[blastbeng]

I don't think it has anything to do with wrong IP, Portainer in the past months had been incompatible with Docker 26 exactly in the Console section and only a couple of weeks ago it's been fixed, but seems not so well. I never had problems with that before, when a component was missing I could simply apt install it and use it. I have 75 containers, all working fine, the only one not working is yours: allow me to doubt I have a networking problem. Now, while Navidrome is still scanning the library after deploying the merged compose, Subtify still gives the same error

2024-09-12 19:33:21 ERROR    Job "artist_reccomendations (trigger: interval[1:00:00], next run at: 2024-09-12 20:33:21 CEST)" raised an exception
Traceback (most recent call last):
  File "/home/user/.local/lib/python3.10/site-packages/apscheduler/executors/base.py", line 125, in run_job
    retval = job.func(*job.args, **job.kwargs)
  File "/home/user/subtify/./main.py", line 145, in artist_reccomendations
    generate_playlists.show_recommendations_for_artist(random.choice(generate_playlists.get_artists_array_names()))
  File "/home/user/subtify/./generate_playlists.py", line 41, in get_artists_array_names
    artists = pysonic.getArtists()
  File "/home/user/.local/lib/python3.10/site-packages/libsonic/connection.py", line 1611, in getArtists
    res = self._doInfoReq(req)
  File "/home/user/.local/lib/python3.10/site-packages/libsonic/connection.py", line 2720, in _doInfoReq
    res = self._opener.open(req)
  File "/usr/local/lib/python3.10/urllib/request.py", line 525, in open
    response = meth(req, response)
  File "/usr/local/lib/python3.10/urllib/request.py", line 634, in http_response
    response = self.parent.error(
  File "/usr/local/lib/python3.10/urllib/request.py", line 563, in error
    return self._call_chain(*args)
  File "/usr/local/lib/python3.10/urllib/request.py", line 496, in _call_chain
    result = func(*args)
  File "/usr/local/lib/python3.10/urllib/request.py", line 643, in http_error_default
    raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 405: Method Not Allowed

Navidrome works fine, Subtify in the same compose does not. And the script says:

root@openmediavault:~/subtify# ./first_run.sh
Traceback (most recent call last):
  File "init.py", line 1, in <module>
    import spotipy
ImportError: No module named spotipy

That looks very strange and difficult to replicate

About the script error. Are you sure you execute the script inside the container? Because the container user is "user" and not root

Anyway if you merge the two compose file and put both under the same network you maybe need to point the subsonic ip to the name of the navidrome container.

Post me your actual merged compose file, I will modify it for you and we will try again.

And another question, which version of navidrome are you using? It logs the version at startup. And it is under a sub path like http://192.168.1.x:4533/navidrome or just http://192.168.1.x:4533?

[janaxhell]

I forgot to paste the script from docker

root@openmediavault:/srv/dev-disk-by-uuid-5b67514d-485e-4306-873e-b1cbb54ccf99/Config/subtify# docker compose exec -it subtify /home/user/subtify/first_run.sh
no configuration file provided: not found

The internal path of navidrome is http://192.168.1.94:4533/music

This is the merged compose:

services:
    navidrome:
        container_name: navidrome
        image: deluan/navidrome:latest
        user: 998:100
        ports:
            - "4533:4533"
        environment:
            ND_SCANSCHEDULE: 5 6 * * 0
            ND_LOGLEVEL: info  
            ND_SESSIONTIMEOUT: 72h
            ND_BASEURL: "/music"
            ND_PLAYLISTSPATH: "lidarr/playlists"
            # ND_IMAGECACHESIZE: "100MB"
            ND_SPOTIFY_ID: XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
            ND_SPOTIFY_SECRET: XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
            ND_LASTFM_ENABLED: true
            ND_LASTFM_APIKEY: XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
            ND_LASTFM_SECRET: XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
            ND_SUBSONICARTISTPARTICIPATIONS: true
            ND_LISTENBRAINZ_BASEURL: http://192.168.1.94:42010/apis/listenbrainz/1/
            ND_UIWELCOMEMESSAGE: Be water my friend
            ND_ENABLEREPLAYGAIN: true
            ND_ENABLESHARING: true
            ND_ENABLESTARRATING: true
            ND_IGNOREDARTICLES: "The El La Los Las Le Les Os As O A Il Lo La Gli"
            ND_ENABLEFAVOURITES: true
            ND_ENABLEEXTERNALSERVICES: true
            ND_ENABLECOVERANIMATION: true             
            ND_COVERARTPRIORITY: embedded, folder.*, cover.*, front.*
            ND_JUKEBOX_ENABLED: true
        volumes:
            - "/srv/dev-disk-by-uuid-5b67514d-485e-4306-873e-b1cbb54ccf99/Config/Navidrome:/data"
            - "/srv/dev-disk-by-uuid-BAF04088F0404D37/data/media/unmapped:/music/unmapped"
            - "/srv/dev-disk-by-uuid-BAF04088F0404D37/data/media/music:/music/lidarr"
        restart: unless-stopped

    subtify:
        container_name: subtify
        environment:
            - PUID=998
            - PGID=100
            - TZ=Europe/Rome
            - SPOTIPY_CLIENT_ID=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
            - SPOTIPY_CLIENT_SECRET=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
            - SPOTIPY_REDIRECT_URI=http://127.0.0.1:18080/
            - SUBSONIC_API_HOST=http://192.168.1.94
            - SUBSONIC_API_PORT=4533
            - SUBSONIC_API_USER=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
            - SUBSONIC_API_PASS=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
        image: "blastbeng/subtify:latest"
        user: 998:100
        restart: always
        volumes:
            - "/srv/dev-disk-by-uuid-5b67514d-485e-4306-873e-b1cbb54ccf99/Config/subtify/cache:/home/user/subtify/cache"
        ports:
            - 50811:50811
networks:
    omv1:
      external: true 

[blastbeng]

I forgot to paste the script from docker

root@openmediavault:/srv/dev-disk-by-uuid-5b67514d-485e-4306-873e-b1cbb54ccf99/Config/subtify# docker compose exec -it subtify /home/user/subtify/first_run.sh
no configuration file provided: not found

The internal path of navidrome is http://192.168.1.94:4533/music

This is the merged compose:

services:
    navidrome:
        container_name: navidrome
        image: deluan/navidrome:latest
        user: 998:100
        ports:
            - "4533:4533"
        environment:
            ND_SCANSCHEDULE: 5 6 * * 0
            ND_LOGLEVEL: info  
            ND_SESSIONTIMEOUT: 72h
            ND_BASEURL: "/music"
            ND_PLAYLISTSPATH: "lidarr/playlists"
            # ND_IMAGECACHESIZE: "100MB"
            ND_SPOTIFY_ID: XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
            ND_SPOTIFY_SECRET: XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
            ND_LASTFM_ENABLED: true
            ND_LASTFM_APIKEY: XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
            ND_LASTFM_SECRET: XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
            ND_SUBSONICARTISTPARTICIPATIONS: true
            ND_LISTENBRAINZ_BASEURL: http://192.168.1.94:42010/apis/listenbrainz/1/
            ND_UIWELCOMEMESSAGE: Be water my friend
            ND_ENABLEREPLAYGAIN: true
            ND_ENABLESHARING: true
            ND_ENABLESTARRATING: true
            ND_IGNOREDARTICLES: "The El La Los Las Le Les Os As O A Il Lo La Gli"
            ND_ENABLEFAVOURITES: true
            ND_ENABLEEXTERNALSERVICES: true
            ND_ENABLECOVERANIMATION: true             
            ND_COVERARTPRIORITY: embedded, folder.*, cover.*, front.*
            ND_JUKEBOX_ENABLED: true
        volumes:
            - "/srv/dev-disk-by-uuid-5b67514d-485e-4306-873e-b1cbb54ccf99/Config/Navidrome:/data"
            - "/srv/dev-disk-by-uuid-BAF04088F0404D37/data/media/unmapped:/music/unmapped"
            - "/srv/dev-disk-by-uuid-BAF04088F0404D37/data/media/music:/music/lidarr"
        restart: unless-stopped

    subtify:
        container_name: subtify
        environment:
            - PUID=998
            - PGID=100
            - TZ=Europe/Rome
            - SPOTIPY_CLIENT_ID=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
            - SPOTIPY_CLIENT_SECRET=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
            - SPOTIPY_REDIRECT_URI=http://127.0.0.1:18080/
            - SUBSONIC_API_HOST=http://192.168.1.94
            - SUBSONIC_API_PORT=4533
            - SUBSONIC_API_USER=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
            - SUBSONIC_API_PASS=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
        image: "blastbeng/subtify:latest"
        user: 998:100
        restart: always
        volumes:
            - "/srv/dev-disk-by-uuid-5b67514d-485e-4306-873e-b1cbb54ccf99/Config/subtify/cache:/home/user/subtify/cache"
        ports:
            - 50811:50811
networks:
    omv1:
      external: true 

1) The init command is wrong, you used docker compose exec, but the right one is "docker exec"

2) you configured a network in your compose file but forgot to add this network to navidrome and subtify config

3) There is an issue in the code, you have navidrome under a supath, not like me. I have not implemented this yet. I will fix in the next release v 0.1.1 in an hour or so and post a correct compose file.

[janaxhell]

1. Docker compose : my bad, i blindly copypasted the very first command i found in the readme this morning
2. Networks: ok, I never merge composes, I'm not used to add the info multiple times, will do now, I suppose it's networks: omv1 at the end of both parts
3. /music: I deployed Navidrome a couple of years ago, I can't remember if it was mandatory at the time, but never cared to remove it. EDIT I have a problem with networks: omv1entry, what indentation do I need?

Failed to deploy a stack: validating /data/compose/2/docker-compose.yml: services.navidrome.networks must be a list

Nevermind, got it, it's

        networks: 
            - omv1

[blastbeng]

1. Docker compose : my bad, i blindly copypasted the very first command i found in the readme this morning
2. Networks: ok, I never merge composes, I'm not used to add the info multiple times, will do now, I suppose it's networks: omv1 at the end of both parts
3. /music: I deployed Navidrome a couple of years ago, I can't remember if it was mandatory at the time, but never cared to remove it. EDIT I have a problem with networks: omv1entry, what indentation do I need?

Failed to deploy a stack: validating /data/compose/2/docker-compose.yml: services.navidrome.networks must be a list

Nevermind, got it, it's

        networks: 
            - omv1

You can remove that "/music", it is not mandatory.

I have settled the ND_BASEURL as "" in my compose file.

If you remove that, without using docker networks, all should magically start working

Anyway sorry about the issue, but it is still a work in progress.

[janaxhell]

You can remove that "/music", it is not mandatory.

I have settled the ND_BASEURL as "" in my compose file.

I may try, but I prefer to leave it alone, as i have several devices configured with that.

If you remove that, without using docker networks, all should magically start working

What do you mean by this? omv1? I can't, no more non-overlapping IPs and me too dumb to fix it.

Anyway sorry about the issue, but it is still a work in progress.

No problem at ALL!! It will be a very cool tool when it starts rolling :)

[blastbeng]

You can remove that "/music", it is not mandatory. I have settled the ND_BASEURL as "" in my compose file.

I may try, but I prefer to leave it alone, as i have several devices configured with that.

If you remove that, without using docker networks, all should magically start working

What do you mean by this? omv1? I can't, no more non-overlapping IPs and me too dumb to fix it.

Anyway sorry about the issue, but it is still a work in progress.

No problem at ALL!! It will be a very cool tool when it starts rolling :)

No worries, just wait and in an hour or so I will release version 0.1.1 with the fix :) You'll need to force pull the latest image

[blastbeng]

Released VERSION 0.1.1

Stop the container, completely remove the image and repull from scratch

@janaxhell I have tested everything using your modified compose file with some fixes from myself, if you use this exact compose file you should be good now:

services:
    navidrome:
        container_name: navidrome
        image: deluan/navidrome:latest
        user: 998:100
        networks:
            - ndsubtifynet
        ports:
            - "4533:4533"
        environment:
            ND_SCANSCHEDULE: 5 6 * * 0
            ND_LOGLEVEL: info  
            ND_SESSIONTIMEOUT: 72h
            ND_BASEURL: "/music"
            ND_PLAYLISTSPATH: "lidarr/playlists"
            # ND_IMAGECACHESIZE: "100MB"
            ND_SPOTIFY_ID: XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
            ND_SPOTIFY_SECRET: XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
            ND_LASTFM_ENABLED: true
            ND_LASTFM_APIKEY: XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
            ND_LASTFM_SECRET: XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
            ND_SUBSONICARTISTPARTICIPATIONS: true
            ND_LISTENBRAINZ_BASEURL: http://192.168.1.94:42010/apis/listenbrainz/1/
            ND_UIWELCOMEMESSAGE: Be water my friend
            ND_ENABLEREPLAYGAIN: true
            ND_ENABLESHARING: true
            ND_ENABLESTARRATING: true
            ND_IGNOREDARTICLES: "The El La Los Las Le Les Os As O A Il Lo La Gli"
            ND_ENABLEFAVOURITES: true
            ND_ENABLEEXTERNALSERVICES: true
            ND_ENABLECOVERANIMATION: true             
            ND_COVERARTPRIORITY: embedded, folder.*, cover.*, front.*
            ND_JUKEBOX_ENABLED: true
        volumes:
            - "/srv/dev-disk-by-uuid-5b67514d-485e-4306-873e-b1cbb54ccf99/Config/Navidrome:/data"
            - "/srv/dev-disk-by-uuid-BAF04088F0404D37/data/media/unmapped:/music/unmapped"
            - "/srv/dev-disk-by-uuid-BAF04088F0404D37/data/media/music:/music/lidarr"
        restart: unless-stopped

    subtify:
        container_name: subtify
        environment:
            - PUID=998
            - PGID=100
            - TZ=Europe/Rome
            - SPOTIPY_CLIENT_ID=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
            - SPOTIPY_CLIENT_SECRET=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
            - SPOTIPY_REDIRECT_URI=http://127.0.0.1:18080/
            - SUBSONIC_API_BASE_URL=/music
            - SUBSONIC_API_HOST=http://navidrome
            - SUBSONIC_API_PORT=4533
            - SUBSONIC_API_USER=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
            - SUBSONIC_API_PASS=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
            - ITEMS_PER_PLAYLIST=100
            - NUM_USER_PLAYLISTS=5
            - ARTIST_GEN_SCHED=2
            - RECCOMEND_GEN_SCHED=8
            - PLAYLIST_GEN_SCHED=6
            - SAVED_GEN_SCHED=24
            - SCHEDULER_ENABLED=1
            - SPOTDL_ENABLED=0
            - LOG_LEVEL=20
        image: "blastbeng/subtify:latest"
        user: 998:100
        restart: always
        volumes:
            - "/srv/dev-disk-by-uuid-5b67514d-485e-4306-873e-b1cbb54ccf99/Config/subtify/cache:/home/user/subtify/cache"
        networks:
            - ndsubtifynet
        ports:
            - 50811:50811

networks:
  ndsubtifynet:

If you want you can rename the network name, but leave the network configuration as is As you can see there is a new env variable "SUBSONIC_API_BASE_URL=/music" this must be the same subpath as ND_BASEURL without the double quotes

[janaxhell]

Hi, last night I went offline. This morning I've deployed the new version from scratch, looking at the logs it seems OK, but after more than 1 hour nothing happened. I've used exactly the same values of your compose. Yesterday when it wasn't connecting, it did do stuff every 1 hour, now it's just sitting. Do I just have to wait more? In the homepage there is this note

NOTE. Depending on your library size and your playlists number and size on Spotify, the execution may take a very long time. To avoid Spotify rate limiting a lot of time.sleep() have ben added to the code.

Maybe a message or counter would help to know what it's doing/that is doing something. I don't hear any "scan noise" coming from the harddisk. Also, will this be long the same every time it starts a job, or cache previous data? I'm asking because when Lidarr and Emby trigger, they scan the entire library, and myself too when I curate the new additions from MusicBee, so I'd like to minimize the overlapping of these tasks.

*** Operational MODE: single process ***
*** uWSGI is running in multiple interpreter mode ***
spawned uWSGI master process (pid: 7)
spawned uWSGI worker 1 (pid: 8, cores: 1)
spawned uWSGI http 1 (pid: 9)
░██████╗██╗░░░██╗██████╗░████████╗██╗███████╗██╗░░░██╗
██╔════╝██║░░░██║██╔══██╗╚══██╔══╝██║██╔════╝╚██╗░██╔╝
╚█████╗░██║░░░██║██████╦╝░░░██║░░░██║█████╗░░░╚████╔╝░
░╚═══██╗██║░░░██║██╔══██╗░░░██║░░░██║██╔══╝░░░░╚██╔╝░░
██████╔╝╚██████╔╝██████╦╝░░░██║░░░██║██║░░░░░░░░██║░░░
╚═════╝░░╚═════╝░╚═════╝░░░░╚═╝░░░╚═╝╚═╝░░░░░░░░╚═╝░░░
                              v0.1.1 
2024-09-13 09:43:18 INFO     Adding job tentatively -- it will be properly scheduled when the scheduler starts
2024-09-13 09:43:18 INFO     Adding job tentatively -- it will be properly scheduled when the scheduler starts
2024-09-13 09:43:18 INFO     Adding job tentatively -- it will be properly scheduled when the scheduler starts
2024-09-13 09:43:18 INFO     Adding job tentatively -- it will be properly scheduled when the scheduler starts
2024-09-13 09:43:18 INFO     Added job "artist_reccomendations" to job store "default"
2024-09-13 09:43:18 INFO     Added job "my_reccommendations" to job store "default"
2024-09-13 09:43:18 INFO     Added job "user_playlists" to job store "default"
2024-09-13 09:43:18 INFO     Added job "saved_tracks" to job store "default"
2024-09-13 09:43:18 INFO     Scheduler started
WSGI app 0 (mountpoint='') ready in 1 seconds on interpreter 0x559474449ee0 pid: 8 (default app)

[blastbeng]

Hi, last night I went offline. This morning I've deployed the new version from scratch, looking at the logs it seems OK, but after more than 1 hour nothing happened. I've used exactly the same values of your compose. Yesterday when it wasn't connecting, it did do stuff every 1 hour, now it's just sitting. Do I just have to wait more? In the homepage there is this note

NOTE. Depending on your library size and your playlists number and size on Spotify, the execution may take a very long time. To avoid Spotify rate limiting a lot of time.sleep() have ben added to the code.

Maybe a message or counter would help to know what it's doing/that is doing something. I don't hear any "scan noise" coming from the harddisk. Also, will this be long the same every time it starts a job, or cache previous data? I'm asking because when Lidarr and Emby trigger, they scan the entire library, and myself too when I curate the new additions from MusicBee, so I'd like to minimize the overlapping of these tasks.

*** Operational MODE: single process ***
*** uWSGI is running in multiple interpreter mode ***
spawned uWSGI master process (pid: 7)
spawned uWSGI worker 1 (pid: 8, cores: 1)
spawned uWSGI http 1 (pid: 9)
░██████╗██╗░░░██╗██████╗░████████╗██╗███████╗██╗░░░██╗
██╔════╝██║░░░██║██╔══██╗╚══██╔══╝██║██╔════╝╚██╗░██╔╝
╚█████╗░██║░░░██║██████╦╝░░░██║░░░██║█████╗░░░╚████╔╝░
░╚═══██╗██║░░░██║██╔══██╗░░░██║░░░██║██╔══╝░░░░╚██╔╝░░
██████╔╝╚██████╔╝██████╦╝░░░██║░░░██║██║░░░░░░░░██║░░░
╚═════╝░░╚═════╝░╚═════╝░░░░╚═╝░░░╚═╝╚═╝░░░░░░░░╚═╝░░░
                              v0.1.1 
2024-09-13 09:43:18 INFO     Adding job tentatively -- it will be properly scheduled when the scheduler starts
2024-09-13 09:43:18 INFO     Adding job tentatively -- it will be properly scheduled when the scheduler starts
2024-09-13 09:43:18 INFO     Adding job tentatively -- it will be properly scheduled when the scheduler starts
2024-09-13 09:43:18 INFO     Adding job tentatively -- it will be properly scheduled when the scheduler starts
2024-09-13 09:43:18 INFO     Added job "artist_reccomendations" to job store "default"
2024-09-13 09:43:18 INFO     Added job "my_reccommendations" to job store "default"
2024-09-13 09:43:18 INFO     Added job "user_playlists" to job store "default"
2024-09-13 09:43:18 INFO     Added job "saved_tracks" to job store "default"
2024-09-13 09:43:18 INFO     Scheduler started
WSGI app 0 (mountpoint='') ready in 1 seconds on interpreter 0x559474449ee0 pid: 8 (default app)

This software doesn't scan anything at boot, it scans your spotify and navidrome library on a timer when needed. I haven't implemented a spotify scan at boot because of spotify rate limiting. The first run is after 1 hour, as you can see in the docs:

Name	Info	Default	Mandatory
ARTIST_GEN_SCHED	Interval in hours to schedule the artists reccomendations generation, set to 0 to disable this generator	1	No
RECCOMEND_GEN_SCHED	Interval in hours to schedule the custom reccomendations generation, set to 0 to disable this generator	4	No
PLAYLIST_GEN_SCHED	Interval in hours to schedule the custom playlist import, set to 0 to disable this generator	3	No
SAVED_GEN_SCHED	Interval in hours to schedule the saved tracks playlist import, set to 0 to disable this generator	2	No

this means:

• artist reccomendations every 1 hour
• your custom reccomendations every 4 hours
• playlist import every 3 hours
• saved track playlist import every 2 hours

You can modify these parameters in your compose file.

Or if you just want to test this out go to: http://YOURIP:50811/ and manually execute some of the APIs:

FLASK APP ENDPOINTS:

• /generate/artist_reccomendations// => Generate artist reccomendations playlists, if no artist is provided it will choose a random one from your library
• /generate/artist_reccomendations/all/ => Generate reccomendations playlists for all the artists in your library
• /generate/reccomendations => Generate a random reccomendations Playlist
• /import/user_playlists => import a random playlist from your spotify account
• /import/user_playlists/all => import all playlist from your spotify account
• /import/saved_tracks => import a playlist with all your saved tracks from your spotify account

EDIT: OH and i forgot. This log is that long because you have LOG_LEVEL=20 in the compose file. If you want less logging, set it as 40.

- LOG_LEVEL=40

[janaxhell]

Ok, I have manually triggered using the http://yourip:50811/generate/artist_reccomendations/ and this rather misleading message appeared.

But then in Navidrome a new playlist appeared, although a little weird: it's called L'Aura - Reccomendations and contains completely unrelated artists

Is that Spotify that associates "L'Aura" to those?

This software doesn't scan anything at boot, it scans your spotify and navidrome library

You mean it scans just the navidrome.db file?

[blastbeng]

Ok, I have manually triggered using the http://yourip:50811/generate/artist_reccomendations/ and this rather misleading message appeared.

But then in Navidrome a ne playlist appeared, although a little weird: it's called L'Aura - Reccomendations and contains completely unrelated artists

Is that Spotify that associates "L'Aura" to those?

This software doesn't scan anything at boot, it scans your spotify and navidrome library

You mean it scans just the navidrome.db file?

The error is ok, it's because your browser doesn't support the response type. The correct way to launch the rest apis is from: http://yourip:50811/

The Playlist name is correct and it is not weird. When it generates artist reccomendations it generates a playlist named "ARTIST_NAME - Reccomendation". Look my navidrome:

000 - Saved Tracks is you spotify saved tracks playlist, imported from spotify 00x -My Reccomendation is one of the five custom playlist generated by your top tracks from spotify Artist - Reccomendations is an artist playlist reccomendations, imported from spotify PLAYLIST_NAME is a playlist created and imported from spotify

Everything is documented in the readme file, take a look at it https://github.com/blastbeng/subtify/blob/main/README.md

Anyway it doesn't scan the navidrome.db file. it connect to navidrome using subsonic rest APIs and search for songs in your library.

[janaxhell]

No, I meant I don't think L'Aura is similar to the artists in the generated playlist. Anyway, looking at the logs, I see many songs considered not found that are 100% present in my library

 WARNING  Track Sheryl Crow - Everyday Is A Winding Road not found in your music library
2024-09-13 12:08:51 INFO     Searching Amy Macdonald - Mr Rock & Roll in your music library
2024-09-13 12:08:52 WARNING  Track Amy Macdonald - Mr Rock & Roll not found in your music library
2024-09-13 12:08:52 INFO     Searching fun. - Some Nights in your music library
2024-09-13 12:08:54 WARNING  Track fun. - Some Nights not found in your music library
2024-09-13 12:08:54 INFO     Searching Norah Jones - Sunrise in your music library
2024-09-13 12:08:55 WARNING  Track Norah Jones - Sunrise not found in your music library
2024-09-13 12:08:55 INFO     Searching George Michael - Freedom! '90 - Remastered in your music library
2024-09-13 12:08:57 WARNING  Track George Michael - Freedom! '90 - Remastered not found in your music library
2024-09-13 12:08:57 INFO     Searching Kate Bush - Running Up That Hill (A Deal With God) in your music library
2024-09-13 12:08:58 WARNING  Track Kate Bush - Running Up That Hill (A Deal With God) not found in your music library
2024-09-13 12:08:58 INFO     Searching Seal - Kiss from a Rose in your music library
2024-09-13 12:09:00 WARNING  Track Seal - Kiss from a Rose not found in your music library
2024-09-13 12:09:00 INFO     Searching Of Monsters and Men - Mountain Sound in your music library
2024-09-13 12:09:01 WARNING  Track Of Monsters and Men - Mountain Sound not found in your music library
2024-09-13 12:09:01 INFO     Searching Train - Drive By in your music library
2024-09-13 12:09:03 WARNING  Track Train - Drive By not found in your music library
2024-09-13 12:09:03 INFO     Searching Eliza Doolittle - Pack Up in your music library
2024-09-13 12:09:04 WARNING  Track Eliza Doolittle - Pack Up not found in your music library
2024-09-13 12:09:04 INFO     Searching The Police - Every Little Thing She Does Is Magic in your music library
2024-09-13 12:09:05 WARNING  Track The Police - Every Little Thing She Does Is Magic not found in your music library
2024-09-13 12:09:05 INFO     Searching Paloma Faith - Never Tear Us Apart in your music library

How does it parse the library? I have a heavily curated library with metadata mostly from Musicbrainz. EDIT Is it maybe because I have /music/GENRE/artist/albums? And also /unmapped/GENRE/artist/albums to keep stuff that Lidarr doesn't recognize separated, but present in Navidrome.

[blastbeng]

No, I meant I don't think L'Aura is similar to the artists in the generated playlist. Anyway, looking at the logs, I see many songs considered not found that are 100% present in my library

This is spotify fault, I insert in those playlists what spotify send me.

 WARNING  Track Sheryl Crow - Everyday Is A Winding Road not found in your music library
2024-09-13 12:08:51 INFO     Searching Amy Macdonald - Mr Rock & Roll in your music library
2024-09-13 12:08:52 WARNING  Track Amy Macdonald - Mr Rock & Roll not found in your music library
2024-09-13 12:08:52 INFO     Searching fun. - Some Nights in your music library
2024-09-13 12:08:54 WARNING  Track fun. - Some Nights not found in your music library
2024-09-13 12:08:54 INFO     Searching Norah Jones - Sunrise in your music library
2024-09-13 12:08:55 WARNING  Track Norah Jones - Sunrise not found in your music library
2024-09-13 12:08:55 INFO     Searching George Michael - Freedom! '90 - Remastered in your music library
2024-09-13 12:08:57 WARNING  Track George Michael - Freedom! '90 - Remastered not found in your music library
2024-09-13 12:08:57 INFO     Searching Kate Bush - Running Up That Hill (A Deal With God) in your music library
2024-09-13 12:08:58 WARNING  Track Kate Bush - Running Up That Hill (A Deal With God) not found in your music library
2024-09-13 12:08:58 INFO     Searching Seal - Kiss from a Rose in your music library
2024-09-13 12:09:00 WARNING  Track Seal - Kiss from a Rose not found in your music library
2024-09-13 12:09:00 INFO     Searching Of Monsters and Men - Mountain Sound in your music library
2024-09-13 12:09:01 WARNING  Track Of Monsters and Men - Mountain Sound not found in your music library
2024-09-13 12:09:01 INFO     Searching Train - Drive By in your music library
2024-09-13 12:09:03 WARNING  Track Train - Drive By not found in your music library
2024-09-13 12:09:03 INFO     Searching Eliza Doolittle - Pack Up in your music library
2024-09-13 12:09:04 WARNING  Track Eliza Doolittle - Pack Up not found in your music library
2024-09-13 12:09:04 INFO     Searching The Police - Every Little Thing She Does Is Magic in your music library
2024-09-13 12:09:05 WARNING  Track The Police - Every Little Thing She Does Is Magic not found in your music library
2024-09-13 12:09:05 INFO     Searching Paloma Faith - Never Tear Us Apart in your music library

How does it parse the library? I have a heavily curated library with metadata mostly from Musicbrainz.

He parse the library using song artist and song name. But like I said before this is a work in progress and some songs may not be found yet.

Anyways this is my log from today (saved tracks) , I almost have everything and the log is this:

subtify    | 2024-09-13 10:51:53 INFO     Searching Punkreas - Déjà-Vu in your music library
subtify    | 2024-09-13 10:51:53 INFO     Track Punkreas - Déjà-Vu found in your music library
subtify    | 2024-09-13 10:51:53 INFO     Searching Punkreas - I Signori Della Guerra in your music library
subtify    | 2024-09-13 10:51:53 INFO     Track Punkreas - I Signori Della Guerra found in your music library
subtify    | 2024-09-13 10:51:53 INFO     Searching Punkreas - Disagio (feat. Raphael) in your music library
subtify    | 2024-09-13 10:51:54 INFO     Track Punkreas - Disagio (feat. Raphael) found in your music library
subtify    | 2024-09-13 10:51:54 INFO     Searching Raphael - Disagio (feat. Raphael) in your music library
subtify    | 2024-09-13 10:51:54 INFO     Track Raphael - Disagio (feat. Raphael) found in your music library
subtify    | 2024-09-13 10:51:54 INFO     Searching Punkreas - Giorno Perfetto in your music library
subtify    | 2024-09-13 10:51:54 INFO     Track Punkreas - Giorno Perfetto found in your music library
subtify    | 2024-09-13 10:51:54 INFO     Searching Punkreas - Uomo Medioevo in your music library
subtify    | 2024-09-13 10:51:54 INFO     Track Punkreas - Uomo Medioevo found in your music library
subtify    | 2024-09-13 10:51:54 INFO     Searching Punkreas - Il Prossimo Show - Electric Version in your music library
subtify    | 2024-09-13 10:51:54 WARNING  Excluding search result Punkreas - Il Prossimo Show - Acoustic - Funny Goes Acoustic because it contains excluded words
subtify    | 2024-09-13 10:51:54 WARNING  Excluding search result Punkreas - Il Prossimo Show - Acoustic - Funny Goes Acoustic because it contains excluded words
subtify    | 2024-09-13 10:51:54 WARNING  Excluding search result Punkreas - Il Prossimo Show - Acoustic - Il Prossimo Show (Acoustic) because it contains excluded words
subtify    | 2024-09-13 10:51:54 WARNING  Excluding search result Punkreas - Il Prossimo Show - Acoustic - Il Prossimo Show (Acoustic) because it contains excluded words
subtify    | 2024-09-13 10:51:54 INFO     Track Punkreas - Il Prossimo Show - Electric Version found in your music library
subtify    | 2024-09-13 10:51:54 INFO     Searching Punkreas - Battaglia Persa in your music library
subtify    | 2024-09-13 10:51:55 INFO     Track Punkreas - Battaglia Persa found in your music library
subtify    | 2024-09-13 10:51:55 INFO     Searching Punkreas - Tempi Distorti in your music library
subtify    | 2024-09-13 10:51:55 INFO     Track Punkreas - Tempi Distorti found in your music library
subtify    | 2024-09-13 10:51:55 INFO     Searching Punkreas - Non C'è Più Tempo in your music library
subtify    | 2024-09-13 10:51:55 INFO     Track Punkreas - Non C'è Più Tempo found in your music library
subtify    | 2024-09-13 10:51:55 INFO     Searching Punkreas - Dai Dai Dai (Die Die Die) (feat. Giancane) in your music library
subtify    | 2024-09-13 10:51:55 INFO     Track Punkreas - Dai Dai Dai (Die Die Die) (feat. Giancane) found in your music library
subtify    | 2024-09-13 10:51:55 INFO     Searching Giancane - Dai Dai Dai (Die Die Die) (feat. Giancane) in your music library
subtify    | 2024-09-13 10:51:56 INFO     Track Giancane - Dai Dai Dai (Die Die Die) (feat. Giancane) found in your music library
subtify    | 2024-09-13 10:51:56 INFO     Searching Punkreas - Le Mani In Alto in your music library
subtify    | 2024-09-13 10:51:56 INFO     Track Punkreas - Le Mani In Alto found in your music library
subtify    | 2024-09-13 10:51:56 INFO     Searching Punkreas - Ciao Ciao in your music library
subtify    | 2024-09-13 10:51:56 INFO     Track Punkreas - Ciao Ciao found in your music library
subtify    | 2024-09-13 10:51:56 INFO     Searching Punkreas - Modena - Milano in your music library
subtify    | 2024-09-13 10:51:56 INFO     Track Punkreas - Modena - Milano found in your music library
subtify    | 2024-09-13 10:51:56 INFO     Searching Modena City Ramblers - Modena - Milano in your music library
subtify    | 2024-09-13 10:51:56 INFO     Track Modena City Ramblers - Modena - Milano found in your music library
subtify    | 2024-09-13 10:51:56 INFO     Searching Punkreas - Dai Dai Dai (Die Die Die) (feat. Giancane) in your music library
subtify    | 2024-09-13 10:51:57 INFO     Track Punkreas - Dai Dai Dai (Die Die Die) (feat. Giancane) found in your music library
subtify    | 2024-09-13 10:51:57 INFO     Searching Giancane - Dai Dai Dai (Die Die Die) (feat. Giancane) in your music library
subtify    | 2024-09-13 10:51:57 INFO     Track Giancane - Dai Dai Dai (Die Die Die) (feat. Giancane) found in your music library
subtify    | 2024-09-13 10:51:57 INFO     Searching AC/DC - Fire Your Guns in your music library
subtify    | 2024-09-13 10:51:57 WARNING  Track AC/DC - Fire Your Guns not found in your music library
subtify    | 2024-09-13 10:51:57 INFO     Searching Skid Row - Youth Gone Wild in your music library
subtify    | 2024-09-13 10:51:57 WARNING  Track Skid Row - Youth Gone Wild not found in your music library
subtify    | 2024-09-13 10:51:57 INFO     Searching Warrant - Cherry Pie in your music library
subtify    | 2024-09-13 10:51:58 WARNING  Track Warrant - Cherry Pie not found in your music library
subtify    | 2024-09-13 10:51:58 INFO     Searching The Offspring - Make It All Right in your music library
subtify    | 2024-09-13 10:51:58 WARNING  Track The Offspring - Make It All Right not found in your music library
subtify    | 2024-09-13 10:51:58 INFO     Searching Corey Taylor - Beyond in your music library
subtify    | 2024-09-13 10:51:58 WARNING  Track Corey Taylor - Beyond not found in your music library
subtify    | 2024-09-13 10:51:58 INFO     Searching Alestorm - Drink in your music library
subtify    | 2024-09-13 10:51:58 INFO     Track Alestorm - Drink found in your music library
subtify    | 2024-09-13 10:51:58 INFO     Searching Serj Tankian - Sky Is Over in your music library
subtify    | 2024-09-13 10:51:58 INFO     Track Serj Tankian - Sky Is Over found in your music library
subtify    | 2024-09-13 10:51:58 INFO     Searching Punkreas - Il mondo in your music library
subtify    | 2024-09-13 10:51:59 INFO     Track Punkreas - Il mondo found in your music library
subtify    | 2024-09-13 10:51:59 INFO     Searching Piotta - Il mondo in your music library
subtify    | 2024-09-13 10:51:59 WARNING  Track Piotta - Il mondo not found in your music library
subtify    | 2024-09-13 10:51:59 INFO     Searching Punkreas - Il Mercato Del Niente in your music library
subtify    | 2024-09-13 10:51:59 INFO     Track Punkreas - Il Mercato Del Niente found in your music library
subtify    | 2024-09-13 10:51:59 INFO     Searching Punkreas - Battaglia Persa in your music library
subtify    | 2024-09-13 10:51:59 INFO     Track Punkreas - Battaglia Persa found in your music library
subtify    | 2024-09-13 10:51:59 INFO     Searching Finley - I MIEI AMICI (feat. Punkreas) in your music library

EDIT Is it maybe because I have /music/GENRE/artist/albums?

no.

I am implementing better track matching and word exclusion in the next release planned for today.

Anyways we're going offtopic now, the python error looks solved and you're correctly running subtify. If you need any other help just PM me or add me on discord.