search

jellyfin / mopidy-jellyfin

Jellyfin Extension for Mopidy
https://jellyfin.org
Apache License 2.0
94 stars 16 forks source link

Not all Music & Playlists Shown in M.A.L.P. & Web Frontends #18

Closed feerlessleadr closed 4 years ago

feerlessleadr commented 4 years ago

Hi - I was able to sort out all of my previous issues, and now have the latest version of mopidy running using the local extension, which plays my music great.

I've now moved on to getting jellyfin setup, and while it looks like everything connects ok, when I view my Jellyfin Library in either the Musicbox webclient, or M.A.L.P. app, I only see a very limited number of artist folders (16 folders out of 225 that are on my server). I also don't see anyway to view my playlists which I have imported into jellyfin.

Jellyfin shows all of the music & playlist files correctly.

I turned on verbose logging, restarted mopidy, and then using the musicbox webclient, browsed Jellyfin showing only the 16 folders. The log is too long to paste here, so please see below:

https://paste.ubuntu.com/p/dRmnWc8kQt/

My ultimate goal is to be able to use Jellyfin to manage my music and playlists, rather than mopidy local.

Thanks!

mcarlton00 commented 4 years ago

That's super odd. Is your library in Jellyfin one Music folder or so you have several folders associated to the same library? Is your account allowed to access all libraries? I think playlists are still a bit broken under limited permissions until the JF database rework lands.

I'm not sure how to translate this into the service and config file, but you can put the plugin into debug logging mode like this:

mopidy -v -o loglevels/mopidy=info -o loglevels/jellyfin=debug

This will print out the json directly as it's received from the server. Be warned that it's a lot and isn't the easiest to read right now.

feerlessleadr commented 4 years ago

Thanks - nothing special on my JF library. I have one folder for music (that has subfolders by artist and album), and I have the drive mounted directly to my ubuntu server. I'm assuming that it isn't a permissions issue, since I can see some of the folders and play the music within those folders, but not all.

I ran that command with the following change:

mopidy -v -o loglevels/mopidy=info -o loglevels/jellyfin=debug

but it fills up so much of my ssh terminal window, that I can't copy most of the output. I've tried redirecting the output to a file, but for some reason it won't save to the file.

In the end though, my main reason for trying to get this set up was for playlist support, so sounds like this might be a moot point anyway

mcarlton00 commented 4 years ago

Playlists definitely work, I'm just not understanding why both playlists and only pieces of your music library aren't appearing.

alt text

In the next day or two I can put together another short python script that will print out a list of artists and playlists from the Jellyfin API. It might give us a better idea of why it's not reading some things from your server correctly

feerlessleadr commented 4 years ago

Awesome thanks!

mcarlton00 commented 4 years ago

Alright, give this a try. It requires the requests library be installed, but that's a dependency for mopidy-jellyfin so that shouldn't be a problem for you. It will print out a list of all artists found, and then print a list of all playlists found.

import requests

server_url = 'http://192.168.0.10:8096' # Change me
username = 'user' # Change me
password = 'password' # Change me

auth_data = {
    'Username': username,
    'Pw': password
}

headers = {}

authorization = (
    'MediaBrowser , '
    'Client="other", '
    'Device="script", '
    'DeviceId="script", '
    'Version="0.0.0"'
)

headers['x-emby-authorization'] = authorization

r = requests.post(server_url + '/Users/AuthenticateByName', headers=headers, data=auth_data)

token = r.json().get('AccessToken')
userid = r.json().get('User').get('Id')

headers['x-mediabrowser-token'] = token

r = requests.get(server_url + '/Artists', headers=headers)
artists = r.json().get('Items')

print('===== Found Artists =====')
for item in artists:
    print(item.get('Name'))

r = requests.get(server_url + '/Users/' + userid + '/Items?Recursive=true&IncludeItemTypes=Playlist', headers=headers)
playlists = r.json().get('Items')

print('\n\n===== Found Playlists =====')
for item in playlists:
    print(item.get('Name'))

Example output:

===== Found Artists =====
.38 Special
"Weird Al" Yankovic
2Cellos
3 Doors Down
10 Years
11 Acorn Lane
......

===== Found Playlists =====
Acoustic Rock
Instrumental Focus
Rock
Melodic Beat
feerlessleadr commented 4 years ago

Thanks - looks like it found everything (although that doesn't say much given my taste in music lol)

kevin@linuxserver:~/mopidy$ python3 jellyfintest.py
===== Found Artists =====
AC/DC
Aerosmith
Akinyele
Alice in Chains
The All-American Rejects
America
American Hi-Fi
The Animals
The Ataris
Audioslave
Avicii
The Band
Barenaked Ladies
The Beatles
Beck
Better Than Ezra
Big Audio Dynamite II
Billy Joel
Biz Markie
Black Sabbath
Blake Shelton
Blind Melon
blink-182
blink‐182
Bloodhound Gang
Blue Öyster Cult
Blues Traveler
Bon Jovi
Boston
Bowling For Soup
Brett Eldredge
Brian Fallon
Bruce Springsteen
Bryan Adams
Buffalo Springfield
Bush
Canaan Smith
Candlebox
Carrie Underwood
Cassadee Pope
Cee-Lo Green
CFO$
Chantal Kreviazuk
Cheap Trick
Chevelle
Chris Lane
Chris Young
Circa Waves
The Clash
Claude-Michel Schönberg
Coldplay
Cole Swindell
Collective Soul
Counting Crows
Cracker
Creedence Clearwater Revival
Darius Rucker
Daughtry
David Lee Roth
Deep Blue Something
Def Leppard
Disturbed
Don McLean
The Doors
Dropkick Murphys
The Dubliners
Dustin Lynch
Eagles
Electric Light Orchestra
Eli Young Band
Elisa
Elton John
Eric Church
Europe
Eve 6
Everclear
Fall Out Boy
Fastball
Finger Eleven
Five for Fighting
Fleetwood Mac
Flogging Molly
Florida Georgia Line
Foo Fighters
Foreigner
Fountains of Wayne
Frank Turner
Frankie Ballard
Free
fun.
Garth Brooks
The Gaslight Anthem
Golden Earring
Goldfinger
Goo Goo Dolls
Good Charlotte
Green Day
Guns N' Roses
Harvey Danger
Hoobastank
Hootie & the Blowfish
Imagine Dragons
Incubus
Iron Maiden
Israel Kamakawiwoʻole
Jake Owen
James
Janelle Monáe
Jason Aldean
Jeff
Jeff Buckley
The Jimi Hendrix Experience
Jimmy Eat World
Joe Cocker
John Fogerty
John Legend
John Mellencamp
Johnny Cash
Jon Pardi
Josh Groban
Journey
Justin Moore
Kansas
Karen Fairchild
Keith Urban
Kelly Clarkson
Kenny Chesney
Kid Rock
Kings of Leon
Kip Moore
KISS
Lady Antebellum
Lee Brice
Lenny Kravitz
Linkin Park
Lit
Little Big Town
Live
Luke Bryan
Lustra
Lynyrd Skynyrd
Maddie & Tae
Manowar
Maren Morris
The Marshall Tucker Band
Meat Loaf
Men Without Hats
Metallica
Michael Bublé
Modern English
The Moldy Peaches
Mumford & Sons
My Chemical Romance
Mychael Danna
Neil Diamond
New Found Glory
New Radicals
Nickelback
Nine Days
Oasis
The Offspring
Old Dominion
Panic! at the Disco
Papa Roach
Pat Benatar
Pearl Jam
Phillip Phillips
Pink Floyd
Poison
The Proclaimers
Queen
R.E.M.
Radiohead
Rage Against the Machine
Red Hot Chili Peppers
Reel Big Fish
The Rolling Stones
The Romantics
Rufus Wainwright
Sam Cooke
Sam Hunt
Semisonic
Sharon Shannon
Simon & Garfunkel
Simple Plan
Smash Mouth
Snow Patrol
Social Distortion
Soundgarden
Spin Doctors
SR-71
Staind
Steppenwolf
Steve Earle
Steve Miller Band
Stone Temple Pilots
Styx
Sublime
Sugarcult
Sum 41
Taking Back Sunday
Tenacious D
Thin Lizzy
Third Eye Blind
Thomas Rhett
A Thousand Horses
Tim McGraw
Tom Petty
Tom Petty and the Heartbreakers
Tomoyasu Hotei
Train
Trans-Siberian Orchestra
Tyler Farr
Van Halen
Van Morrison
Vertical Horizon
The Verve Pipe
Victoria Police Pipe Band
Violent Femmes
The Wallflowers
Weezer
Whitesnake
The Who
Willie Nelson
Wolfmother
Yellowcard
Zac Brown Band
Zack Hemsey
The Zombies
|

===== Found Playlists =====
Recently Added
Top Rated
Awesome Inc.
Country
mcarlton00 commented 4 years ago

Well, it's finding playlists, so they should be populating into mopidy and the frontend.

Also I lied (sorry). The MusicBox frontend is specifically using the browse functionality instead of the built in mopidy library. So it's actually using a different api call, because *reasons. Try this script instead to see what the browse function can see in the server. In theory, it should match up pretty closely to what your file structure looks like on the disk of your music library.

import requests

server_url = 'http:192.168.0.10:8096' # Change me
username = 'user' # Change me
password = 'password' # Change me
library_name = 'Music' # Maybe Change me?

auth_data = {
    'Username': username,
    'Pw': password
}

headers = {}

authorization = (
    'MediaBrowser , '
    'Client="other", '
    'Device="script", '
    'DeviceId="script", '
    'Version="0.0.0"'
)

headers['x-emby-authorization'] = authorization

r = requests.post(server_url + '/Users/AuthenticateByName', headers=headers, data=auth_data)

token = r.json().get('AccessToken')
userid = r.json().get('User').get('Id')

headers['x-mediabrowser-token'] = token

r = requests.get(server_url + '/Users/' + userid + '/Views', headers=headers)
libraries = r.json().get('Items')

music_id = [ x.get('Id') for x in libraries if x.get('Name') == library_name][0]

contents = requests.get(server_url + '/Users/{}/Items?ParentId={}&SortOrder=Ascending'.format(userid, music_id), headers=headers).json()

print('===== Top level contents of "' + library_name + '" =====')
for item in contents.get('Items'):
    print(item.get('Name'))

*The reasons are because of incompatibilities between the API and different media types (namely music and book libraries). I'd like to rework this at some point, but haven't found the right method to do it yet.

feerlessleadr commented 4 years ago

Thanks - it looks like all my artist folders show up, although they are out of order, similar to what shows up in musicbox (way more though). No playlists show up. BTW - the server url was missing 2 forward dashes, I added them in, and I get the below.

Should I try out a different front end?

kevin@linuxserver:~/mopidy$ python3 jellyfintest2.py
===== Top level contents of "Music" =====
.stfolder
Phillip Phillips
Playlists
Biz Markie
Brian Fallon
CFO$
Chantal Kreviazuk
Imagine Dragons ft. Elisa
Imagine Dragons
Rufus Wainwright
Jeff Danna
Sharon Shannon
Steve Earle
The Dubliners
Fountains of Wayne
Frank Turner
Frankie Ballard
Free
fun_
Golden Earring
Goldfinger
Guns N' Roses
Hoobastank
Hootie & the Blowfish
Iron Maiden
Jake Owen
Jimmy Eat World
Five for Fighting
Fleetwood Mac
Blue Öyster Cult
Blues Traveler
Boston
Johnny Cash
Keith Urban
Live
Luke Bryan
Lynyrd Skynyrd
Manowar
Maren Morris
Metallica
Modern English
Mumford & Sons
John Legend
Jon Pardi
Journey
Kings of Leon
KISS
Lee Brice
Linkin Park
Lit
Simple Plan
Smash Mouth
Snow Patrol
Social Distortion
SR-71
Staind
Styx
Sublime
Sugarcult
Tenacious D
The All-American Rejects
The Animals
The Beatles
The Gaslight Anthem
The Jimi Hendrix Experience
The Moldy Peaches
The Offspring
Train
Trans-Siberian Orchestra
Incubus
James
Jason Aldean
Joe Cocker
Kip Moore
Lady Antebellum
Lenny Kravitz
Old Dominion
Semisonic
Tom Petty
Tom Petty and the Heartbreakers
Victoria Police Pipe Band
Zac Brown Band
Zack Hemsey
Jeff Buckley
John Fogerty
Soundgarden
Steppenwolf
Steve Miller Band
Stone Temple Pilots
Sum 41
Taking Back Sunday
The Ataris
The Clash
The Doors
The Rolling Stones
The Wallflowers
The Zombies
Thin Lizzy
Third Eye Blind
Thomas Rhett
Tim McGraw
Tyler Farr
Van Halen
Van Morrison
Weezer
Willie Nelson
Wolfmother
John Mellencamp
Justin Moore
Kenny Chesney
Little Big Town
Maddie & Tae
Meat Loaf
New Radicals
Pink Floyd
Sam Hunt
Simon & Garfunkel
Spin Doctors
The Band
Violent Femmes
Whitesnake
Yellowcard
The Marshall Tucker Band
The Proclaimers
The Romantics
The Verve Pipe
Vertical Horizon
The Who
Kill Bill: Vol. 1
Sam Cooke
Pearl Jam
Green Day
America
American Hi-Fi
Avicii
Barenaked Ladies
Beck
Candlebox
Cee-Lo Green
Chevelle
Chris Young
Counting Crows
Cracker
Darius Rucker
Deep Blue Something
Dustin Lynch
Eve 6
Everclear
Fall Out Boy
Fastball
Finger Eleven
Foreigner
Garth Brooks
Goo Goo Dolls
Harvey Danger
Aerosmith
Jacqueline
Chris Lane
Israel Kamakawiwoʻole
Josh Groban
Kansas
Lustra
Men Without Hats
Michael Bublé
Electric Light Orchestra
Elton John
Eric Church
Europe
Flogging Molly
Florida Georgia Line
Foo Fighters
Good Charlotte
A Thousand Horses
Audioslave
Billy Joel
Black Sabbath
Blake Shelton
blink-182
Bon Jovi
Carrie Underwood
Eli Young Band
Neil Diamond
New Found Glory
Nickelback
Nine Days
Oasis
Panic! at the Disco
Papa Roach
Pat Benatar
Poison
Queen
R.E.M_
Radiohead
Rage Against the Machine
Red Hot Chili Peppers
Reel Big Fish
Ac_Dc
Akinyele
Alice in Chains
Better Than Ezra
Big Audio Dynamite II
Blind Melon
Bloodhound Gang
My Chemical Romance
Brett Eldredge
Buffalo Springfield
Bowling For Soup
Bruce Springsteen
Bryan Adams
Bush
Canaan Smith
Cheap Trick
Claude-Michel Schönberg
Coldplay
Cole Swindell
Collective Soul
Creedence Clearwater Revival
Daughtry
David Lee Roth
Def Leppard
Disturbed
Don McLean
Dropkick Murphys
Eagles
mcarlton00 commented 4 years ago

Yeah at this point I'm at a loss as to what could be going on. If you're remotely comfortable with the terminal, try ncmpcpp. That's my main client and what I do most of my dev work against.

Once it's open, 2 will bring you to the browse menu, and 5 will bring you to the playlist view If we're living in a sane world, then when you go to 2 -> Jellyfin -> Music then it should match your filesystem layout (which should also match what's in MusicBox, but currently isn't for reasons unknown).

If ncmpcpp works, then it sounds like there's something screwy with your install of MusicBox somehow. If it doesn't work, then I'm largely out of ideas. If weirdness continues, would you be open to making a temporary account on your server for me so I can try with your files? I can keep sending options to try various things, but being hands on will speed up the process if there's a real bug hidden in here.

feerlessleadr commented 4 years ago

Sorry for the late reply - work has been a little nuts, so haven't had a chance to mess with ncmpcpp. I will though this weekend and will report back.

If all else fails, I will throw my jellyfin instance on the net and will give you a login to test.

mcarlton00 commented 4 years ago

@feerlessleadr have you had a chance to look at this again?

sudotm commented 4 years ago

I am experiencing this identical issue and would be happy to provide whatever assistance I can to help squash this bug. Some observations are below, let me know if any of this needs more clarification.

Same as @feerlessleadr experienced, only a small portion of my music library shows up when using the mopidy front-end's File Browser (20 out of 1000+) and the built-in Mopidy library remains empty. I have tried on several front-ends (Musicbox, Iris, Cantata, MALP and ncmpcpp) and each exhibit the same issues.

I have ran both scripts you pasted above, and they report back what seems to be a full list of artists.

I've noticed that the only items that do show up in the mopidy library are things that were never processed through my Beets autotagger and do not have a Musicbrainz ID associated with it in Jellyfin. Not all the material without a Musicbrainz ID appears, just some of it.

Below is a sample of what I see with when attempting to browse the Jellyfin folder inside of mopidy:

Jellyfin
|_ Music
  |_ Artist 1
    |_ tracks
  |_ Artist 2
    |_ (empty)
  |_ Album 2
    |_ tracks
  |_ Album 3
    |_ tracks
  |_ Artist 4
    |_ empty

Most of my library is organized Artist -> Album -> Tracks, but a few odd things are organized Artist -> Tracks on the actual music server and each one of those odd items do show up in mopidy. This can be seen above with Artist 1.

The empty folder shown with Artist 2 is where Album 2 is supposed to be nested, but Album 2 instead shows up next to it's parent folder instead of inside of it. Album 3 also is nested inside an Artist folder on the actual music server, but the parent folder in this case is not displayed by mopidy. Similarly, there are some empty Artist folders without the cooresponding Album folder.

My music library works fine with Jellyfin normally. Casting content to the mopidy device from Jellyfin also works without issue, no matter whether or not those files can be seen from the mopidy interface.

Mopidy-local doesn't seem to experience this issue either, though I was never actually able to complete a full local scan due to the music library being relatively large. I am also running mpd on another box (not mopidy-mpd) and that also works without issue on my entire library.

I have installed mopidy and the jellyfin extension per the instructions in the jellyfin documentation on a freshly formatted Rasbian Lite. I am on the newest stable version of Jellyfin server. I have tried tweaking the config file (albumartistsort, http/https hostname) but get the same result each time.

MALP shows "Error 2/0: Incorrect Arguments" when attempting to load the database and that is echoed in the debug log with the lines:

DEBUG 2020-08-19 00:22:27,172 [1366:MpdSession-23] mopidy_mpd.session Response to [::ffff:192.168.1.110]:38273: ACK [2@0] {list} incorrect arguments

I see nothing else of interest from the debug log.

I hope some of this ends up being useful. My guess is the few badly tagged/organized items in the library is messing up the entire thing. Sort of like one bad apple spoils the entire bunch. I am going to experiment on a smaller copy of my music library with only properly tagged content to see how that goes and hopefully pinpoint what makes things fall apart.

sudotm commented 4 years ago

Update: I created a new, small music library in JF with just a few well-tagged items organized with the folder structure Artists -> Albums -> Tracks, nothing appears in Mopidy. Moving all the Album folders up a level so the folder structure is just Albums -> Tracks, then everything shows up in the Mopidy file browser. Though, the built-in Mopidy library still remains empty and I continue to receive the "Incorrect Argument" error when loading it with MALP.

I also took a closer look at my full library and confirmed things only show up when there are tracks down a single level from the music folder, so Album -> Tracks structure. The odd behavior I described above of empty artist folders as well as album folders appearing next to the artist folder instead of inside of them seem to occur when the folder structure looks like so:

Artist folder
  |_ Track1.mp3
  |_Track2.mp3
  |_ Album folder
    |_Track1.mp3
    |_ etc

Somehow this causes weirdness and the Album folder ends up one directory above where it should be. The part that is the most strange is that if the folder structure is perfectly organized with Artist folder -> Album folder -> Tracks then nothing shows up at all.

mcarlton00 commented 4 years ago

Well something weird is definitely going on, but I'm having the opposite results as you.

ZZ Ward
├── 365_Days_
│   └── ZZ_Ward-Blue_Eyes_Blind_(Live_From_Fairfax_Recordings).m4a
├── The Storm (2017)
│   ├── ZZ Ward - 01 - Ghost.mp3
│   ├── ZZ Ward - 02 - Cannonball.mp3
│   ├── ZZ Ward - 03 - Help Me Mama.mp3
└── Til the Casket Drops
    ├── 01 Til the Casket Drops.mp3
    ├── 02 Put the Gun Down.mp3
    ├── 03 Blue Eyes Blind.mp3
    ├── 04 Home.mp3
    ├── 05 Cryin Wolf.mp3

works fine, but 

Imagine Dragons
├── 01 Ready Aim Fire.mp3
├── Believer
│   └── 01 Believer.mp3
├── Night Visions
│   ├── 01 Radioactive.mp3
│   ├── 03 It’s Time.mp3
│   ├── 04 Demons.mp3
│   ├── 05 On Top of the World.mp3
│   └── 09 Bleeding Out.mp3

doesn't seem to be. Very strange. But at least that's something concrete I can work from now

mcarlton00 commented 4 years ago

I think I made a breakthough in relation to this. The problem is likely a combination of 2 things:

Now the reason it behaves differently for these two functions is wanting to support Audiobooks as well. They don't have a dedicated endpoint and don't work properly under /Artists (for somewhat obvious reasons). There's likely a better way to work through this, though I'm not sure what that would look like yet. More thought is going to be needed

mcarlton00 commented 4 years ago

In the process of reworking the browse function, I found a bug in the server that seems to be the real root cause of all this. Basically, having a file structure like in the previous comment causes the server to not link tracks to the right albums (or not create albums at all). See https://github.com/jellyfin/jellyfin/issues/3962 for more details. The TLDR is make sure you music library is sorted by

Artist
  Album1
    Tracks
  Album2
    Tracks

if you want everything to get populated and be browsable correctly. I have some mitigation to make sure that artists are visible and browsable here, but I'm not going to go real in depth to get the albums displaying correctly when that fix needs to happen upstream.