Some Watchlist "Problems"

[dominikheiss]

Hey guys,

I managed to get everything working in my UNRAID environment, despite the fact, that it's throwing errors from the Watchlists others users using my Plex server (does this only work for "local users"? Also my watchlist is being created, but not copied to cache :( Don't know where to start sadly...I hope someone can support me :)

I also couldn't use my public dns name for my plex server (even valid certificates and stuff, but it told me "token is malformed"...)

[bexem]

Hello! I'm sorry you're experiencing some issues...

it's throwing errors from the Watchlists others users using my Plex server

I've recently noticed an access error with the watchlist feature. Upon checking, I found that the updated plexapi now requires the api call to specify when it's using a token, instead of merely passing the token as before.

(does this only work for "local users"?)

No, it fetches all users.

I also couldn't use my public dns name for my plex server (even valid certificates and stuff, but it told me "token is malformed"...)

I only utilize the external domain for Plex. Due to my intentional firewall settings, I can't access Plex using the internal IP from the server running the script. Therefore, any issues related to domain access shouldn't originate from the code but I can't rule it out completely, I base myself on the fact no other user is experiencing it and during my quick review of the code it still uses up-to-date plexapi calls.

Also my watchlist is being created, but not copied to cache

This is odd, I'm guessing from the size of the file you actually have a list of media inside?

Try with the updated code, let me know, if it still doesn't work, run it with debug on and send me the log, I'll try and have a look when possible.

[dominikheiss]

Thanks for answering that fast ;)

I updated your script, deleted the watchlist_cache and rerun the script with debugging enabled: plexcache_log_20230823_1941.log: https://pastebin.com/b51U0pcb

plexcache_watchlist_cache.json is filled with my stuff (no other users media) with a correctly edited path. The media can also be found on my server.

The DNS topic does not matter... I was only thinking if that could be related. I'm using Chronos Docker with a host network adapter and when I add: "PLEX_URL": "https://plex.mydomain.level/",

I get this:

But like I said, I'd be more than happy if the watch list and other users would work ;)

[bexem]

With chronos I had this kind of issues, literally randomly, sometimes I just had to reboot the container, sometimes change the network of the container. That's why I stopped using it and advising it as the best way to run the script.

Regarding the error, I've updated the code once again, I'm not sure why it gives you that error. The script practically fetches all the users in your server, and then iterate each one of them to fetch the watchlist. The fact the fetches "random_username1" and then says it's not found, it doesn't make sense... What I thought is the script might have issues with complex usernames and titles, so now it fetches all the users and if they have a username it uses that rather than the title...according to the documentation, it shouldn't matter, but worth a shot. I've also added a delay, so that it doesn't spam the plexapi when requesting the watchlist for many users.

[dominikheiss]

Okay, understood.... I think I will change from Chronos to User-Scripts then.

But sadly it still does not catch the other users watchlist....only my data gets populated in the file, BUT also not copied, even though available.

https://pastebin.com/wW4c1tsQ

[bexem]

Try directly with userscripts, if it still gives the same error I’ll have a look again.

On Wed, 23 Aug 2023 at 20:10, dominikheiss @.***> wrote:

Okay, understood.... I think I will change from Chronos to User-Scripts then.

But sadly it still does not catch the other users watchlist....only my data gets populated in the file, BUT also not copied, even though available.

https://pastebin.com/wW4c1tsQ

— Reply to this email directly, view it on GitHub https://github.com/bexem/PlexCache/issues/21#issuecomment-1690499925, or unsubscribe https://github.com/notifications/unsubscribe-auth/ADDVY32LUYQL54QDPZ4PVVDXWZISXANCNFSM6AAAAAA33GLMNA . You are receiving this because you commented.Message ID: @.***>

--

[dominikheiss]

I just did and sadly it's exactly the same :( I'm trying some more things but as everything besides watchlists is working I don't think it's a configuration error....but who knows ;)

Thanks for your effort though!

[dominikheiss]

//UPDATE: I got my setup working with using the public DNS name. Involved some fiddling with cloudflare tunnel.

Despite that I still get the errors from the watchlists and no media from the watchlist file gets pulled to the cache. Just tell me if I can support you with troubleshooting ;)

Also one more question: how does the "watched_move" process act, if movieX is on two users decks and one of them watched it. Will it be moved back to the array or only if it is watched from all user's decks?

[bexem]

//UPDATE: I got my setup working with using the public DNS name. Involved some fiddling with cloudflare tunnel.

That's great!

Also one more question: how does the "watched_move" process act, if movieX is on two users decks and one of them watched it. Will it be moved back to the array or only if it is watched from all user's decks?

The common media will be moved only when nobody has it in the watchlist or ondeck, when iterating all the media, the script lists all the files that needs to go the array and to the cache, and before actually moving them, it "compares notes" and skips any files destined to the array if the same file is meant to stay or be moved to the array. (Otherwise you will have a ping pong effect, wasting time and energy).

I still get the errors from the watchlists and no media from the watchlist file gets pulled to the cache.

Can you try this? https://pastebin.com/pctPrGFi

It's the standalone version of the watchlist fetching, it will print the whole fetched media for each user. It will likely give the same error as it's the same code, but at least debugging should be easier.

[dominikheiss]

Here is the output of the file:

https://pastebin.com/gDGYCDN9

I tried to troubleshoot with ChatGPT like 2 hours today, but he is no magician it seems ;)

[bexem]

Here is the output of the file:

https://pastebin.com/gDGYCDN9

I tried to troubleshoot with ChatGPT like 2 hours today, but he is no magician it seems ;)

Yeah I wish it could fix the (whole) code but unfortunately it isn't as capable yet...

I've implemented an error catching, so for the users it cannot "find" it will still continue but in a clean way.

https://pastebin.com/pctPrGFi

I'm having trouble understanding why this issue is occurring. My server is accessed by both local users and external users and both the script and the snippet are retrieving all the watchlist media for all my users. I'm unsure if this is due to a account permission problem or an issue with the APIs themselves. The script/snippet literally fetched the user list, and uses that information to request the watchlist, but when it does it, plex says that the user doesn't exist? I just can't understand why right now.

[dominikheiss]

Here is the output: https://pastebin.com/MaTkswQt

Yes I agree, the error is super weird.

But even if we ignore that it runs into the supposed "API errors", it should read the content of the "plexcache_watchlist_cache.json" (which contains my content - with correct path) and move it to the SSD.

But man, if I am the only one with this problem, don't spend too much time with it. It's more then okay, that the script fetches all the on-deck stuff from every user.

[bexem]

It's because it ends with an error and it doesn't proceed, if I manage a decent error catching when is having this issues, it should then continue and copy whatever it managed to fetch from the watchlist(s).

https://pastebin.com/pctPrGFi I've updated it again, let me know if it still gives error.

[dominikheiss]

Ah okay...understood.

[dominikheiss]

It can't be an error in my settings.json right? I mean with all the path variables and stuff..... if these were the problem, it wouldn't move on-deck stuff to the cache, right?

[bexem]

Well no, you are having the same user error not found even on my test script which doesn't touch the settings files at all. And I have set a fairly decent settings loader to avoid settings error (at least obvious ones).

The last error you had it's entirely on me, I forgot to paste from plexapi.exceptions import NotFound at the beginning of the file on pastebin🤦🏻‍♂️

It should hopefully work now?

[dominikheiss]

https://pastebin.com/ttnFhWbv ;)

[dominikheiss]

Look, that's what I meant:

plexcache_log_20230824_2025.log

---> without touching other users it shows no errors, but does not proceed processing the watchlist, or am I mistaken?

[bexem]

What is the content of the watchlist file? The settings file seems correct and the logs doesn't show any errors.

I've re-updated the code on pastebin, it includes another error catching for BadRequests (too many requests).

I really gotta make myself work on the rewrite of the script so that debugging and improving the script will be way easier.

[dominikheiss]

The watchlist file content seems valid...I checked the paths. https://pastebin.com/DX0L4b9S

The new script is running for 2 minutes, no output yet :)

[bexem]

I've checked the files you have on the watchlist_cache file and they are present in the plexcache log file, it means they are in the list of files that needs to be moved. Debug was on, so obviously it wasn't moving anything.

About the other script running for that long...I guess I went too far with the delay time, just stop it.

[dominikheiss]

Debug was on, so obviously it wasn't moving anything.

Yeah sure, I also did that, but with debug it should calculate the GB of the files it would move. At least that's what I saw in other Debug runs. And even without debug it does not move them :(

Here is the output: https://pastebin.com/GLn5ZkYy

[bexem]

So this file "/mnt/user/data/media/tv/permanent/HD/Carnival Row (2019) [imdb-tt0489974]/Season 1/Carnival Row (2019) - S01E01 - Some Dark God Wakes [WEBRip-1080p][AC3 5.1][DE+EN][x265]-FuN.mkv"

I suppose is not on the cache drive? If not it should definitely move that file.

I've updated the main script including what I've learned from the pastebin, I've also added a listing of all the cache files at the end of the script, right before moving files to the cache, just to have some extra info.

[dominikheiss]

Sadly it isn't:

It sits here:

Log from the updated version: plexcache_log_20230824_2139.log

Here with Debug enabled: plexcache_log_20230824_2141.log

[bexem]

Okay try now, it should work.....................................

[dominikheiss]

It really works - at least for my user ;) The others are still API errored, 401, unauthorized whatever :)

plexcache_log_20230824_2231.log

[bexem]

Yeah unfortunately I haven't came up with a solution for that yet...at least the main user works for you... I'm curious if any other user is experiencing your same issue, either way I will carry on and working on it as well as the zfs issue I've left for a while.

Thank you for your patience :)

[dominikheiss]

I have to thank you!! I'm going to buy you a beer 🍻😂

Von meinem/meiner Galaxy gesendet

-------- Ursprüngliche Nachricht -------- Von: "Emanuele B." @.> Datum: 24.08.23 22:37 (GMT+01:00) An: bexem/PlexCache @.> Cc: Dominik Heiß @.>, Author @.> Betreff: Re: [bexem/PlexCache] Some Watchlist "Problems" (Issue #21)

Yeah unfortunately I haven't came up with a solution for that yet...at least the main user works for you... I'm curious if any other user is experiencing your same issue, either way I will carry on and working on it as well as the zfs issue I've left for a while.

Thank you for your patience :)

— Reply to this email directly, view it on GitHubhttps://github.com/bexem/PlexCache/issues/21#issuecomment-1692373360, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AQWQPWGRSOTZUEIAEEDBP2TXW63Q5ANCNFSM6AAAAAA33GLMNA. You are receiving this because you authored the thread.Message ID: @.***>

[dominikheiss]

What is the ZFS issue, if I may ask? (as my cache disk is a raid1 ZFS array ;) )

[dominikheiss]

I have one more question regarding the watchlist of other users: Does it really work for non-home users? Or did I misinterpret that?

Because the script uses "switchHomeUser" and my users are "remote".

Other user's on-deck stuff works, because it's stored on my actual server, but their watchlist data is like in their own account, hence I get the weird authentication errors and such. --> Just an assumption - I have no clue what I'm talking ;)

[bexem]

What is the ZFS issue, if I may ask? (as my cache disk is a raid1 ZFS array ;) )

There is another open issue, where another user was having trouble with the script moving files from the array to the cache, cannot remember now they have either the array or the cache in ZFS. But if the script is working for you, it means it works as it should always have (python can be very platform agnostic)...that's good I guess 😅

I have one more question regarding the watchlist of other users: Does it really work for non-home users? Or did I misinterpret that?

Because the script uses "switchHomeUser" and my users are "remote".

Other user's on-deck stuff works, because it's stored on my actual server, but their watchlist data is like in their own account, hence I get the weird authentication errors and such. --> Just an assumption - I have no clue what I'm talking ;)

When using the Plex API, the "PlexServer" endpoint allows access to the watchlist of the authenticated (main) user. However, it doesn't provide a direct way to access watchlists of other users.
MyPlexAccount allows it instead, but I have to use plex servers and that's why I have to use internet, and the only way to switch users is "switchHomeUser" at least according to the api documentation. The "on-deck" information is stored on the main server, which is why we can access it without issues. But for watchlists, since the data is account-specific, we're limited to accessing only the main user's watchlist via the "MyPlexAccount" connection.

Either way I have home users and remote users, and the script manages to gather all the watchlist:

Fetching Emanuele Bergamini's watchlist media...
Emanuele Bergamini: /media/tvseries/Strange Planet/Season 1/Strange Planet - S01E01 - The Flying Machine WEBDL-2160p.mkv
Emanuele Bergamini: /media/tvseries/Strange Planet/Season 1/Strange Planet - S01E02 - Greyscale Finger Bandit WEBDL-2160p.mkv
Emanuele Bergamini: /media/tvseries/Strange Planet/Season 1/Strange Planet - S01E03 - Careful Now WEBDL-2160p.mkv
Emanuele Bergamini: /media/tvseries/Strange Planet/Season 1/Strange Planet - S01E04 - The Big Wet WEBDL-2160p.mkv
Emanuele Bergamini: /media/tvseries/Strange Planet/Season 1/Strange Planet - S01E05 - Family, Fandom, Footorb WEBDL-1080p.mkv
Emanuele Bergamini: /media/movies/The Orphanage (2007)/The Orphanage (2007) Bluray-1080p.mkv
Emanuele Bergamini: /media/tvseries/Future Man/Season 1/Future Man - S01E01 - Pilot WEBDL-2160p.mkv
Emanuele Bergamini: /media/tvseries/Future Man/Season 1/Future Man - S01E02 - Herpe - Fully Loaded WEBDL-2160p.mkv
Emanuele Bergamini: /media/tvseries/Future Man/Season 1/Future Man - S01E03 - A Riphole in Time WEBDL-2160p.mkv
Emanuele Bergamini: /media/tvseries/Future Man/Season 1/Future Man - S01E04 - A Fuel's Errand WEBDL-2160p.mkv
Emanuele Bergamini: /media/tvseries/Future Man/Season 1/Future Man - S01E05 - Justice Desserts WEBDL-2160p.mkv
Fetching Guest's (LOCAL/HOME USER) watchlist media...
Guest: /media/movies/Insidious Chapter 5 (2023)/Insidious The Red Door (2023) WEBDL-2160p.mkv
Fetching REDACTED_REMOTEUSER's watchlist media...
Fetching REDACTED_REMOTEUSER's watchlist media...
Fetching REDACTED_REMOTEUSER's watchlist media...
REDACTED_REMOTEUSER: /media/movies/Elemental (2023)/Elemental (2023) WEBDL-2160p.mkv
REDACTED_REMOTEUSER: /media/movies/Life Is Beautiful (1997)/Life Is Beautiful (1997) Bluray-1080p.mkv
REDACTED_REMOTEUSER: /media/movies/The Super Mario Bros. Movie (2023)/The Super Mario Bros. Movie (2023) WEBDL-2160p.mkv
REDACTED_REMOTEUSER: /media/movies/Guardians of the Galaxy Vol. 3 (2023)/Guardians of the Galaxy Vol. 3 (2023) WEBDL-2160p.mkv
REDACTED_REMOTEUSER: /media/movies/Transformers Rise of the Beasts (2023)/Transformers Rise of the Beasts (2023) WEBDL-2160p Proper.mkv
REDACTED_REMOTEUSER: /media/movies/Terminator 2 Judgment Day (1991)/Terminator 2 Judgment Day (1991) Bluray-2160p.mkv
REDACTED_REMOTEUSER: /media/movies/Ghostbusters Afterlife (2021)/Ghostbusters Afterlife (2021) WEBDL-2160p.mkv

At least those are what I understood to be remote users...

[dominikheiss]

Okay okay okay....understood! So it really should work.

I'd have one last "request" :)

What works for me:

• fetches ALL on-deck data (so I have to have users_toggle enabled)
• fetches only my watchlist (which is fine..really...no worries)

What does not work, but would be quite nice:

• watched_move --> this only works, when I set the users_toggle to false, else it reads my watched list and writes it to the file, but ignores the move back to the array

I guess it wants to also get the other users watched state and therefore does not start at all. So if you could separate that in any way? :) Like watched_move_users_toggle or whatever, I could have all the features ;)

[bexem]

That is not how is meant to work... Have a look at the updated version, I might have just forgot to pass a variable to the function. I have not tested it (yet), let me know! You might want to delete the cached file....or use my shiny new argument when running the script: python3 plexcache.py --skip-cache

[dominikheiss]

Mhhhh, sadly it didn't improve anything, more like worse ;)

plexcache_log_20230825_1951.log

It does not even populate my plexcache_watched_cache.json, after I deleted it for the new run :/

[bexem]

Apologies, I should have tested it...I've changed how I passed the variable and it seems to be working fine on my end! Check the last commit!

[dominikheiss]

No problem ;)

The variable error is gone, but now we have the same situation as before:

plexcache_log_20230825_2005.log

[bexem]

All the errors you mean? https://pastebin.com/0DBjrFUp

This an extracted and improved version of the watched fetching...It will fetch all users and should handle unauthorised users like the watchlist one.

[dominikheiss]

Naaah stop. It is soooooo weird ;)

I don't know why, but I just did another Debug-Run and now it has populated the watched file (which it did not the last time) and told me that it would move 100GB to the array. Then I started a non debug-run and now it moves 71GB to the array ;)

But doesn't matter, it moves something back^^

[bexem]

So it works...odd about the two different numbers but I guess is what plex told to the script...Anyways the improved version should not affect much, if you don't mind testing it anyway before I commit it?

[dominikheiss]

Yeah, works!! Thank you very much!

I will test it asap... just need to wait for the 71GB transfer.

[bexem]

So I've tested it quite few times now and improved it a little more, I've also submitted it in the last commit, so you can either try the script directly or the pastebin one, they are virtually the same. By the time you run them, you probably don't have any more watched media to move 😅 (I am also copying 98GB, I do have a longer cache expiry time than most)

[dominikheiss]

haha....yeah took quite a while.

but I also tested the new version and it worked. Ty ;)

[bexem]

Perfect, I'm glad it works. Still baffled about your users situation I will keep investigating!

[dominikheiss]

Sure, just tell me if you want me to test something.

Von meinem/meiner Galaxy gesendet

-------- Ursprüngliche Nachricht -------- Von: "Emanuele B." @.> Datum: 25.08.23 20:49 (GMT+01:00) An: bexem/PlexCache @.> Cc: Dominik Heiß @.>, Author @.> Betreff: Re: [bexem/PlexCache] Some Watchlist "Problems" (Issue #21)

Perfect, I'm glad it works. Still baffled about your users situation I will keep investigating!

— Reply to this email directly, view it on GitHubhttps://github.com/bexem/PlexCache/issues/21#issuecomment-1693793947, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AQWQPWEPIBWESYUD7PDSPETXXDXURANCNFSM6AAAAAA33GLMNA. You are receiving this because you authored the thread.Message ID: @.***>

[dominikheiss]

You are also using Unraid, right?

I have the issue, that ur script is running with root privileges and messing with my permissions in the copy process ;) Do you know how I can keep the "nobody" owner of files/folders with your script?

[bexem]

I am using unraid as well, and I did encounter some issues with the permissions but the script is not the only one messing around with it and I have a secondary script to make sure the permissions are always correct in the media folders...

But anyway you are right and the script should not mess with the permissions and rather keep them as they are.

I've updated the code, I've quickly run it but because there is nothing to move I can't tell you for sure how well it works. Might not be relevant for you, but for anyone else reading, it should handle windows as well.

[dominikheiss]

Thanks for the fast response and help - seems to work as it should ;) Have a nice weekend!!

[bexem]

Hey there, I hope everything is still working fine! Unfortunately I've not managed to fix your users error, but another user on reddit was having some issues with the folder creation in the version of the script we have been editing (and you, probably still using). Just wanted to let you know I've updated it to fix it.

I have also implemented a duplicate check, so it will remove the media file from the source if already present in the destination, avoiding back and forth file moves. I will implement a setting for it eventually!

[dominikheiss]

Hey, yeah everything's cool, thanks for asking ;) Thanks for the update, will test it asap!

[dominikheiss]

Hey, I don't know if it's the new version or some randomness, but I got an error, which prevents the function itsself. plexcache_log_20230830_1827.log

I disabled watchlist for now, but does not change anything.

[bexem]

Uhm that's odd, just to be sure, the script you are using, when did you get it? Because I did commit 5h ago, just want to be sure it is the latest one you are using