Replicated #31 - Not syncing movies

[Psu798076]

Just posted this under the closed ticket, and wasn't sure if you'd see it. #31 So I think I have the same problem that @daveonkels had previously - and it appears to be due to having multiple folders in each library (as for me, I had the issue with TV Shows as well). I have two folders for Movies (4k and non-4k) and same for TV. And when it does the search, it only returns the top level. When I modify the query for jellyfin_search to include recursive folders, but filtered for either Movie or Series, it seems to work ok (I think)

jellyfin_search = await self.query( f"/Users/{user_id}/Items" + f"?SortBy=SortName&SortOrder=Ascending&Recursive=true&ParentId={library_id}" + "&isPlayed=false&Fields=ItemCounts,ProviderIds,MediaSources" + f"&IncludeItemTypes=Movie",

jellyfin_search = await self.query( f"/Users/{user_id}/Items" + f"?SortBy=SortName&SortOrder=Ascending&Recursive=true**&ParentId={library_id}" + "&isPlayed=false&Fields=ItemCounts,ProviderIds,Path" + f"&IncludeItemTypes=Series",

[luigi311]

hmm i had recursive on before but didnt notice a difference but then again i do not have two folders for my library like it sounds like you guys do. Ill go ahead and add this back on and make sure it doesnt cause any issues and we can see if it solves these issues.

[Psu798076]

Sorry for the slow response. Awesome - thanks - I did find there were multiple places that had to be adjusted to make it work - I think for the same reasons. I was traveling this week but I’m back home tomorrow and could test any changes you make. Thanks again - great program.

[luigi311]

Can you give me an example of what your library structure is for this. Row now im imagining this by the descriptions

• Movies <- Main Folder/Library Entry
  • 4K Movies
  • 8 Mile (2002)
    • 8 Mile Remux.mkv
  • Movies
  • 8 Mile (2002)
    • 8 Mile Remux.mkv

Where as my folder structure is

• 4K Movies <- Main Folder/Library Entry
  • 8 Mile (2002)
  • 8 Mile Remux.mkv
• Movies <- Main Folder/Library Entry
  • 8 Mile (2002)
  • 8 Mile Remux.mkv

[Psu798076]

Ok, so it’s exactly as you mention. Then I merge the movies using one of the plugins, and so I’m presented with both copies of the movie to play within Emby.Plex, and then depending on if I’m streaming on a 4k TV or not, I choose the appropriate copy.

I used to have it organized like you have yours, but got annoyed that when I would do a search, the movies would show up twice and then I’d be guessing which one was the 4k one and which one was the 1080p copy.

Then the same structure for TV shows.

Does that make sense?

[luigi311]

Ahh yes. So instead of using different libraries you use the play version feature of plex/jellyfin where you can pick between the two from a unified entry.

I actually didnt even consider that since most people i know seperate the library so they dont share the 4k library causing transcodes to plexs default 2mbps lol. Ok ill replicate this structure in my library and give it a try.

[Psu798076]

Yeah, I’ve seen that issue. But mainly that’s just my family members who don’t know what they’re doing, so I ignore them haha….

I could, in theory, go back to the old way (like you had it), but mainly the double searches confused them and annoyed me when they picked the wrong version (like we’re all sitting around watching the movie together and they’re streaming it in 1080p instead of the 4k that was available, because they just don’t care about the difference (whereas i do).

And awesome, thank you for looking into it.

[luigi311]

So im playing with different setups for this, I create a library Movie that has the two folders on it and it seems like that changes the library type from Movie to Folder which explains why the watch list is never generated since i am doing a check for Movie and Series in the get_user_library_watched

[luigi311]

Disregard i was looking at an empty folder without any media in it lol

[luigi311]

I found an issue but not sure if it is the only issue. Getting the library type was done by looking at the type of the first movie/show, in some instances this can be a placeholder folder such as for a movie that is not out yet. Jellyfin for some reason reads in these empty folders and creates entries for them which i wish it would ignore. Instead i am now excluding Folder item types and getting 100 entries and generating a list of types. This also lets me check to see if there are multiple types mixed in to avoid any issues revolving that since i dont have anything implemented for that yet.

[luigi311]

Ok i believe i found all the issues for this. @Psu798076 give this a test with the dev branch when you can and let me know how it goes

[Psu798076]

Ok will do. Will probably be tomorrow morning EST but will let you know.

[Psu798076]

So I get an error (I even tried Blacklisting the library "Live TV" as I wasn't trying to sync it). Haven't looked at the code yet to see why, but figured I'd update this here:

[ERROR]: Jellyfin: Failed to get users watched, Error: Jellyfin: Library Live TV has multiple types: {'Episode', 'Movie'} [ERROR]: Jellyfin: Failed to get watched, Error: 'Exception' object is not iterable [ERROR]: 'Exception' object is not iterable [ERROR]: Traceback (most recent call last): File "/app/src/jellyfin.py", line 435, in get_watched user_watched_temp = combine_watched_dicts(user_watched) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/app/src/functions.py", line 266, in combine_watched_dicts for single_dict in dicts: TypeError: 'Exception' object is not iterable

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/app/src/main.py", line 396, in main main_loop() File "/app/src/main.py", line 341, in main_loop server_2_watched = get_server_watched( ^^^^^^^^^^^^^^^^^^^ File "/app/src/main.py", line 236, in get_server_watched return asyncio.run( ^^^^^^^^^^^^ File "/usr/local/lib/python3.11/asyncio/runners.py", line 190, in run return runner.run(main) ^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/asyncio/runners.py", line 118, in run return self._loop.run_until_complete(task) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/asyncio/base_events.py", line 653, in run_until_complete return future.result() ^^^^^^^^^^^^^^^ File "/app/src/jellyfin.py", line 444, in get_watched raise Exception(e) Exception: 'Exception' object is not iterable

[luigi311]

Yea I need to move the check for multiple library types after the library remover. I also need to update the library remover to handle a list since I think its just for a single type. I'll work on it in a few hours and let you know.

[luigi311]

@Psu798076 Ok give dev another try, it will check if the library should be skipped first prior to checking if multiple types and i also made that skip the library if multiple types instead of just erroring.

[Psu798076]

@luigi311 different error:

[ERROR]: Jellyfin: Failed to get users watched, Error: 'pop from an empty set' [ERROR]: Jellyfin: Failed to get watched, Error: 'Exception' object is not iterable [ERROR]: 'Exception' object is not iterable [ERROR]: Traceback (most recent call last): File "/app/src/jellyfin.py", line 438, in get_watched user_watched_temp = combine_watched_dicts(user_watched) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/app/src/functions.py", line 276, in combine_watched_dicts for single_dict in dicts: TypeError: 'Exception' object is not iterable

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/app/src/main.py", line 396, in main main_loop() File "/app/src/main.py", line 341, in main_loop server_2_watched = get_server_watched( ^^^^^^^^^^^^^^^^^^^ File "/app/src/main.py", line 236, in get_server_watched return asyncio.run( ^^^^^^^^^^^^ File "/usr/local/lib/python3.11/asyncio/runners.py", line 190, in run return runner.run(main) ^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/asyncio/runners.py", line 118, in run return self._loop.run_until_complete(task) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/asyncio/base_events.py", line 653, in run_until_complete return future.result() ^^^^^^^^^^^^^^^ File "/app/src/jellyfin.py", line 447, in get_watched raise Exception(e) Exception: 'Exception' object is not iterable

[luigi311]

Thats weird that you got an empty set. I added in a catch for None and if len of types is not 1 then it will output skipping and output the types list. Give it another try and see if it errors out again or if it shows what the types are.

[Psu798076]

Ok, now I see this: [INFO]: Jellyfin: Skipping Library TV shows not a single type: set().

And then I see[INFO]: matt and None not found in watched list 2.

[Psu798076]

Oh this might be interesting. I think the issue now might be on the Plex side, instead of the Jellyfin side. So, I was whitelisting the user to sync to test to just the user, "Matt", from Plex. That user is a Plex Home user. I just tried syncing the main user account instead of that account, and it seems to be working.

EDIT: Nevermind -same error doing it that way too.

[luigi311]

Can you remove this line on jellyfin if x["Type"] not in ["Folder"]

I think it should be around line 365

[Psu798076]

Done. Ok so now I see this:

[DEBUG]: Jellyfin: Skipping show 4k-TV as it is not in mark list for matt [DEBUG]: Jellyfin: Skipping show TV as it is not in mark list for matt

These are actually the folders but it doesn't seem to be going any deeper to get to the shows or episodes.

[Psu798076]

I should also say I can query the server myself using a tool I have to explore APIs. When I query this (excluded the domain): /Items/?ParentId=28659&Filters=IsPlayed&ExcludeItemTypes=Folder&limit=100 I get those same two folders as listed above.

[Psu798076]

I'm wondering if the remaining issue is specific to TV Shows now. If I do Movies, it seems to mark them all correctly.

[luigi311]

It might be since I only tested with movies and by using the multiple folders method. I'll setup another library on my setup with the same folder structure and I guess I'll get me a 4k TV show so I can actually test it lol

[Psu798076]

Haha ok thanks.

[Psu798076]

@luigi311 so I think I got it to work by making two additional changes:

1. Changed like 340 to: self.query( f"/Users/{user_id}/Items"

   • f"?ParentId={library_id}&Filters=IsPlayed&excludeItemTypes=Folder&limit=100&Recursive=True", "get", session, identifiers=identifiers, )

2. Changed line 365 to: if x["Type"] not in ["Person"]

It seems like in the first query, it was only finding the top level folders so you need to include the recursive stuff. And then in the second part, it seems to be including Person so it was finding two types and then skipping it (based on the logic you implemented). But, I've made a bunch of changes to just try it, so unsure.....

[Psu798076]

Actually I also changed line 156:

watched_shows = await self.query( f"/Users/{user_id}/Items"

• f"?ParentId={library_id}&isPlaceHolder=false&Fields=ProviderIds,Path,RecursiveItemCount&IncludeItemTypes=Series&Recursive=True", "get", session, )

And line 151:

if library_type == "Series":

I think I need to change something on the movies side too

[luigi311]

I think im going to set it to only add in the types if it is Movies or Series instead of excluding options. I will then iterate through the tag list and run the get_user_library_watched for each type and add in that IncludeItemTypes and that recursive. That should let mixed libraries work too since it will grab that library type individually for the libraries, idk why someone would do that but hey i guess its fine lol.

[Psu798076]

Cool, sounds good to me. Will test whenever you let me know. Thanks again for working through this with me. I'm more of a tinkerer than a programmer....

[luigi311]

Ok i added in the changes. I cant seem to test this right now though since my plex server is returning error 500 for some reason on certain libraries.

[Psu798076]

I'm getting an error:

[INFO]: Jellyfin: Skipping Library Movies not a single type: set() [INFO]: Jellyfin: Skipping Library TV shows not a single type: set()

Also, I think you need to somehow exclude the "Person" type because I see this:

[INFO]: Jellyfin: Skipping movie Brendan Donaldson as it is not in mark list for matt

[luigi311]

Hmm that's weird. Can you remove line 365

if x["Type"] in ["Movie", "Series"]

And try again. It should print out all the types for TV shows and movies and iterate through the types so that should generate some watched items.

[Psu798076]

So, I think you still might be missing two spots for recursive (and one filter for Series)

1. Line 342 - needs recursive or else it only returns unwatched top level folders:

   • f"?ParentId={library_id}&Filters=IsPlayed&excludeItemTypes=Folder&limit=100"

2. Line 528/529 - my assumption here is you are trying to retrieve the Series library, the same as you do in the other function, except in the other function you are filtering it by Series and doing it recursively. Here, it's not being filtered (and recursive isn't set to true) and so it's returning everything, including Episodes, Person, etc. So I think you might need to filter this query to IncludeItemTypes=Series. Recursive is also not set in this line.

   • f"?SortBy=SortName&SortOrder=Ascending&Recursive=false&ParentId={library_id}"
     • "&isPlayed=false&Fields=ItemCounts,ProviderIds,Path&excludeItemTypes=Folder",

Once I made these changes, its running fine for TV.

[luigi311]

Ok i added in the recursives in all the places that i think make sense and also switched over to includeitemtypes instead of exclude for anything referencing movies or tv shows. I Think that makes sense since thats all they are designed to handle anyways and if there are other types that are similar it should be as simple as just adding it to the include list. Thanks for your help hopefully this solves it lol

[mjn138]

Seems good now, thanks! As far as I can tell it's all working. Awesome working with you and thanks for creating this set of scripts.

[luigi311]

Perfect thanks for your help @Psu798076 @mjn138 ill go ahead and create a new release with these changes and mark this is as complete. Feel free to reopen it if you have any issues again.

[Psu798076]

Oops, we are one and the same. Must've used the @mjn138 login by accident. Didn't notice until you pointed "both of us" out. Thanks again.