Feeling dumb...

[teshiburu2020]

I'm currently going through the setup.py on this in PowerShell on server 2019, when it gets to the point of asking about my cache drive it defaults to /mnt/cache which is the correct location on my unraid server... but it's not giving me an option to specify any unraid details? have i missed something? or can this not communicate with my unraid server, and will i need to "cache" it on the windows machine locally?

[bexem]

Hi there! The setup script is only to format nicely the settings file, the onlyexternal communication is with the plex server. The cache script need to be run directly on the plexserver machine or at least in a machine who as access to the same paths and able to move files around.

[teshiburu2020]

Then I'll need to be running this from the unraid server directly then as my windows VM for Plex can't see mnt cache....

[bexem]

I suppose so, I personally run it directly on the unraid server (which is also my plex server), I think it makes sense as it has direct access to the directories avoiding any network/protocol overhead when moving files around.

[teshiburu2020]

ok, so i moved it to unraid, and i think my settings is fine:

{
    "PLEX_URL": "http://192.168.1.220:32400",
    "PLEX_TOKEN": "SUPER SECRET",
    "plex_source": "/",
    "plex_library_folders": [
        "P:\\Movies",
        "P:\\TV Shows"
    ],
    "valid_sections": [
        1,
        2
    ],
    "number_episodes": 5,
    "users_toggle": true,
    "skip_users": [
        null,
        null,
        null,
        null,
        null,
        null,
        null
    ],
    "watchlist_toggle": true,
    "watchlist_episodes": 1,
    "watchlist_cache_expiry": 6,
    "watched_cache_expiry": 48,
    "days_to_monitor": 99,
    "cache_dir": "/mnt/cache/",
    "real_source": "/mnt/user/",
    "nas_library_folders": [
        "mnt/MediaShare/Movies",
        "/mnt/MediaShare/TV Shows/"
    ],
    "unraid": true,
    "watched_move": false,
    "skip": true,
    "max_concurrent_moves_cache": 5,
    "max_concurrent_moves_array": 2,
    "debug": false,
    "firststart": false
}

But getting this:

2023-05-19 09:32:00,338 - INFO - Total size of media files to be moved to cache: 0.00 KB
2023-05-19 09:32:00,338 - INFO - Nothing to move to cache
2023-05-19 09:32:00,338 - INFO - Script executed

[bexem]

{
    "PLEX_URL": "https://plex.bexem.xyz",
    "PLEX_TOKEN": "********",
    "plex_source": "/media/",
    "plex_library_folders": [
        "movies",
        "anime",
        "tvseries"
    ],
    "valid_sections": [
        1,
        4,
        2
    ],
    "number_episodes": 10,
    "users_toggle": true,
    "skip_users": [],
    "watchlist_toggle": true,
    "watchlist_episodes": 5,
    "watchlist_cache_expiry": 48,
    "days_to_monitor": 183,
    "watched_move": true,
    "watched_cache_expiry": 12,
    "cache_dir": "/mnt/cache/",
    "real_source": "/mnt/user/",
    "nas_library_folders": [
        "movies",
        "anime",
        "tvseries"
    ],
    "unraid": true,
    "max_concurrent_moves_array": 2,
    "max_concurrent_moves_cache": 5,
    "skip": false,
    "debug": false,
    "firststart": false
}

Your plex library folders and the nas library folder appear to be wrong. Have a look at my settings and compare.

[teshiburu2020]

So is your media in this example located at

/Mnt/user/movies /mnt/user/tvseries

?

Which would suggest I need to change my real source to /mnt/user/mediashare

And then library folders to Movies TV Shows

?

[bexem]

Yes, my movies are in /mnt/user/movies my series are in /mnt/user/tvseries my anime are in /mnt/user/anime

As you can see the real source is /mnt/user/ and the nas_library_folders are the name of the folder above (movies, tvseries and anime)

Also your plex source was wrong at least judging from here, as it has so be the folder the container access the media from inside the container itself, with the plex_library_folders set similarly to the nas folder but according to how you have it configured (mine are exactly the same but you could have movies in the nas folder corresponding to films in the plex library as an example).

[teshiburu2020]

Yeah, the plex is a WINDOWS server not on unraid, well it is its a VM but it access the media on unraid over an NTFS share. so the path for plex media is correct.

However, i think ive found a "windows" specific bug? I've managed to get it to run, and now it gives this error, with the slashes the incorrect way? needs to be some handling for windows file paths as thats how plex sees the media :)

So it now knows what to cache, and where from, but the "replace" isn't flipping the slashes to account for windows

[bexem]

I agree the script should handle that, unfortunately I don’t really have a way to test but I will try my best to fix it anyway. I will probably have a look into it this afternoon. (Night shift, GTM Tz)

Just so it’s clear for me later on: you are running plex on windows directly and not via docker container, correct? Also, where would run the script? UnRaid? If you don’t mind re-sharing your updated setting file?

[teshiburu2020]

I'm running Plex directly on Windows yes.

Unraid is running the script

I think I found a solution which I'll share maybe point you the right way?

[bexem]

Sure, anything can help 😊

[teshiburu2020]

On line 314 I made this change it's crude but seems to have worked

file_path = file_path.replace(folder, nas_library_folders[j]).replace("\\", "/") # Replace the plex library folder with the corresponding NAS library folder

[bexem]

On line 314 I made this change it's crude but seems to have worked

file_path = file_path.replace(folder, nas_library_folders[j]).replace("\\", "/") # Replace the plex library folder with the corresponding NAS library folder

That is actually a not a so dirty solution but I will implement it soon in a conditional manner. Could be unrelated but I rather be sure of it, would you mind telling me how do you have set up the script_folder variable in the cache script? Because you are running it on windows I guess you have a windows formatted folder syntax set?

[teshiburu2020]

I'm running it on unraid

I'll double check on the script folder variable when I'm home but I'm sure I've got it set like... /Mnt/user/system/plexcache or similar

[bexem]

Thank you, I'm working on a universal solution and your case seems perfect for this. So, whenever you can, please attach the redacted settings file you are currently using because I want to be sure the script outputs correctly once I've implemented the fix.

[bexem]

Just an update: I've uploaded a setup_test.py script, it will produce a settings_test.json file (to avoid introducing errors in a working file). I'm leaning towards having the setup script doing the checks and set the paths correctly rather than introducing more logic in the cache script. I might still change my mind, but for testing, the setup script is faster. If you don't mind testing it and posting the result. It might not work at all, I haven't tested it (yet).

Thank you for your time anyway ☺️

[teshiburu2020]

One issue ive noticed in setup.py in both the test and existing is that "watched_cache_expiry" doesn't get generated :)

[bexem]

Well spotted, that was a minor bug that shows up only if answered no to moving the watched media! Fixed! Any other issues?

[teshiburu2020]

OK, so the "settings_test" or something funky is going on.

    "PLEX_URL": "http://192.168.1.220:32400",
    "PLEX_TOKEN": "XXXXXXXX",
    "plex_source": "P:/",
    "plex_library_folders": [
        "P:\\Movies",
        "P:\\TV Shows"
    ],
    "valid_sections": [
        1,
        2
    ],
    "number_episodes": 5,
    "users_toggle": false,
    "skip_users":[],
    "watchlist_toggle": true,
    "watchlist_episodes": 1,
    "watchlist_cache_expiry": 24,
    "watched_cache_expiry": 24,
    "days_to_monitor": 99,
    "cache_dir": "/mnt/cache/MediaShare",
    "real_source": "/mnt/user/MediaShare/",
    "nas_library_folders": [
        "Movies",
        "TV Shows"
    ],
    "unraid": true,
    "watched_move": true,
    "skip": true,
    "max_concurrent_moves_cache": 5,
    "max_concurrent_moves_array": 2,
    "debug": false,
    "firststart": false
}

This is my settings file, it seems to ALWAYS set plex_source as p:/Movies and not just P:/ im assuming a bug here? as it just seems to always put the folder for the "first" library i select here so if i skipped movies and said yes to TV shows it would dbe P:/TV Shows ive been changing this to P:/

But the issue now seems to be that it thinks nothing is to be moved?

2023-05-22 08:17:27,610 - INFO - Initializing script...
2023-05-22 08:17:27,610 - INFO - Checking paths and permissions...
2023-05-22 08:17:27,611 - INFO - Loading watchlist media from cache...
2023-05-22 08:17:27,789 - INFO - Fetching teshiburu's onDeck media...
2023-05-22 08:17:29,083 - INFO - Editing file paths...
2023-05-22 08:17:29,084 - INFO - Fetching subtitles...
2023-05-22 08:17:29,089 - INFO - Loading watched media from cache...
2023-05-22 08:17:29,089 - INFO - Filtering media files array...
2023-05-22 08:17:29,422 - INFO - Total size of media files to be moved to array: 0.00 KB
2023-05-22 08:17:29,422 - INFO - Nothing to move to array
2023-05-22 08:17:29,423 - INFO - Filtering media files cache...
2023-05-22 08:17:29,423 - INFO - Total size of media files to be moved to cache: 0.00 KB
2023-05-22 08:17:29,423 - INFO - Nothing to move to cache
2023-05-22 08:17:29,423 - INFO - Script executed
Thank you for using my script github.com/bexem/PlexCache

[teshiburu2020]

Edit: fixed the cache path above, but that didnt fix the issue, nothing to move apparently?

[bexem]

Edit: fixed the cache path above, but that didnt fix the issue, nothing to move apparently?

Can you share your log file? (With debug mode set to true), it will help debugging.

It's very likely all down to how the script(s) is managing the paths and trying to retrieve the files and checking on the filesytem their existence. I honestly hoped python (as I've read) was going to deal a bit better with paths between windows and posix...but apparently not!

[teshiburu2020]

It seems that no "cache" paths are being generated?

Log.txt

[bexem]

That seems to be the issue, I need to understand why. I'm looking into it...

Not related, but does the script work (well, kind off apparently) if your plex folders are ['Movies', 'TV Shows'] instead of ['P:\Movies', 'P:\TV Shows'] ?

[teshiburu2020]

let me try

same issue, no difference if i remove the P:\

[bexem]

But it still fetches the files correctly?

Either way: The problem is in the fact that is running on linux but is using a paths from windows and linux, I basically need to use a different library to make the script understand what is dealing with first and then correct the paths.

I have no way to test your scenario because both the script and plex server run on linux, I've introduced a plex_cache_test script, it might not work but it's worth a shot. I've also updated the setup_test, if you don't mind retesting that one, please post the exact settings_test.json, just comment how the paths should actually be without correcting it directly. For the plexcache_test, the log should be enough.

Thank you.

[teshiburu2020]

This appears to be generating extra braces?

{}    "PLEX_URL": "http://192.168.1.220:32400",
    "PLEX_TOKEN": 

[teshiburu2020]

ALso seems if i tell it to not import other users it fails to generate this line

"skip_users": [],

which it then moans about :)

[teshiburu2020]

"watched_cache_expiry": 24,

is still missing, watchlist_cache_expiry seems to be generated but not watched

[bexem]

Funny how is giving issues on stuff I haven't even touched 🤣 I'm checking now...

[teshiburu2020]

ok - fixed issues in settings_test.json which i will comment and upload for you :)

but this is the log file after trying to run

2023-05-22 16:03:17,393 - INFO - Editing file paths...
2023-05-22 16:03:17,393 - INFO - Fetching subtitles...
2023-05-22 16:03:17,397 - INFO - Loading watched media from cache...
2023-05-22 16:03:17,397 - INFO - Filtering media files array...
2023-05-22 16:03:17,397 - WARNING - Error: file path P:\TV Shows\A Million Little Things\Season 5\A Million Little Things.-.S05E13.-.one.big.thing.WEBDL-1080p.-.h264 EAC3.mkv is not inside real source path /mnt/user/MediaShare
2023-05-22 16:03:17,397 - INFO - File: P:\TV Shows\A Million Little Things\Season 5\A Million Little Things.-.S05E13.-.one.big.thing.WEBDL-1080p.-.h264 EAC3.mkv
2023-05-22 16:03:17,397 - INFO - Cache_path: None
2023-05-22 16:03:17,397 - INFO - Cache_file_name: None
2023-05-22 16:03:17,397 - ERROR - Error checking free space and moving media files: stat: path should be string, bytes, os.PathLike or integer, not NoneType

[teshiburu2020]

settings_test_upload.txt settings :)

[bexem]

Thank you honestly for testing it, it's a learning experience for me.

So I've updated (once again) the setup script, let me know it works now, should get the operating system on which plex is running automatically, fixed the brakets (honestly, such a weird one!), it should now put the right folders in the settings, and also fixed the cache variables not saving as they should have.

I'm gonna have a look at the cache script in the meantime.

[teshiburu2020]

Awesome I'm out at present I'll check the setup script when I get home.

I may not have the knowledge to do what you are haha, but i can test and click buttons haha

[bexem]

Awesome I'm out at present I'll check the setup script when I get home.

I may not have the knowledge to do what you are haha, but i can test and click buttons haha

Ahahah it works fine for me!

Also, I've updated the plexache test as well, I hope it works but I doubt...I'm not that lucky 🤣 Let me know, as usual the log is the best way to debug the cache script.

[teshiburu2020]

apparently "skip_users":[], is still not being added in when its told to not import other users?

[teshiburu2020]

nor is "watched_cache_expiry":

[teshiburu2020]

{
    "PLEX_URL": "http://192.168.1.220:32400",
    "PLEX_TOKEN": "XXXX",
    "plex_source": "P:\\",
    "plex_library_folders": [
        "P:\\Movies\\",
        "P:\\TV Shows\\"
    ],
    "valid_sections": [
        1,
        2
    ],
    "number_episodes": 5,
    "users_toggle": false,
    "watchlist_toggle": true,
    "watchlist_episodes": 1,
    "watchlist_cache_expiry": 24,
    "days_to_monitor": 99,
    "cache_dir": "\\mnt\\cache\\MediaShare\\",
    "real_source": "\\mnt\\user\\MediaShare\\",
    "nas_library_folders": [
        "\\Movies\\",
        "\\TV Shows\\"
    ],
    "unraid": false,
    "watched_move": true,
    "skip": true,
    "max_concurrent_moves_cache": 5,
    "max_concurrent_moves_array": 2,
    "debug": true,
    "firststart": false
}

Generated settings_test.json

[teshiburu2020]

OK - Tested it all - settings file attached (after ive fixed it commented in the lines i had to add to get it to work)

{
    "PLEX_URL": "http://192.168.1.220:32400",
    "PLEX_TOKEN": "XXXX",
    "plex_source": "P:\\",
    "plex_library_folders": [
        "P:\\Movies\\",
        "P:\\TV Shows\\"
    ],
    "valid_sections": [
        1,
        2
    ],
    "number_episodes": 5,
    "users_toggle": false,
    "skip_users":[],   -- had to add in this line despite "users_toggle" being false
    "watchlist_toggle": true,
    "watchlist_episodes": 1,
    "watchlist_cache_expiry": 24,
    "watched_cache_expiry":24, -- and i had to add this one in 
    "days_to_monitor": 99,
    "cache_dir": "\\mnt\\cache\\MediaShare\\",
    "real_source": "\\mnt\\user\\MediaShare\\",
    "nas_library_folders": [
        "\\Movies\\",
        "\\TV Shows\\"
    ],
    "unraid": false,
    "watched_move": true,
    "skip": true,
    "max_concurrent_moves_cache": 5,
    "max_concurrent_moves_array": 2,
    "debug": true,
    "firststart": false
}

[teshiburu2020]

And I've attached the log file... its fairly large, but from what i can see the "converted" file paths trhat it looks to use are missing the /mnt/user/MediaShare/ portion of their respective names. like "real_source" and "cache_dir" are not being added to the file name somewhere

plexcache_script_latest.log

[teshiburu2020]

plexcache_script_latest (1).log

dropped days to monitor down to1 for you to make the log file smaller ;)

[bexem]

Oddly enough it was adding both watched_cache_expiry and skip_users when tested, but I've issued another fix, it hopefully fixes the paths as well as I am not happy about them. I'm gonna have a look at the logs now.

If the paths are okay in the new settings_test.json might be worth retest the cache script, but probably better to do a step at the time.

[teshiburu2020]

I wish i knew what i was doing to break it between you and me haha but now when it asks me to confirm paths it doesnt tell me for which library i only know because ive done it so much now haha

and the script failed here and didn't proceed further

[bexem]

I hope I get this right this time 😅 Let's try again?

[teshiburu2020]

Still getting the exact same bugs as last night. Fully removed all traces from my server and downloaded fresh from Github

[bexem]

I'm sorry about that, I guess the issue is actually in the conversion. I have not fixed that because I really need to see how it is before and after the conversion, it will print when asking about the plex library before and after, but the logic remained the same. About the array to cache (Just spotted the writing error as well), should be finally fixed now. I must have mixed up between the two enviroments I'm testing 🤦🏻‍♂️

[teshiburu2020]

OK - so is there a fixed version of the setup_test.py? i see a new plexcache_test.py but not setup_test :)

[bexem]

You know the title of this threat? That's me. I don't know what the heck happened with VSCode but it basically replaced the content of the plexcache_test script with the setup_test one...it's fixed now...So now the setup_test is the one I was talking about.

[teshiburu2020]

hahah - oh dear, im sorry to put you through this, have you got a Ko-Fi or a paypal i can drop you a coffee/beer haha!

[teshiburu2020]

not sure if this is right? edit: in hindsight this may be right as its removed the trailing \ ?

also still not giving me a library to confirm the path for :)

something is not right here though hehe - log file was MAHOOSIVE had to zip it up for you as i didn't want to remove anything

plexcache_script_latest.zip

[bexem]

So, the cache script I'm leaving it for the time being because I want to be sure it gets the right paths first (in a specific way) so I can start working in a proper way to fix it. But thank you anyway, I will definitely have a look as it will help me!

Regarding the the setup output: I can see more or less where the issues are now, I just need to make the right logic to format the path.

I've updated it now, I hope it get it right...let me know 🤞🏻🙏🏻

Unfortunately I have to do some other stuff today, but I will still try and work on it later on but I guess we are both on different timezones. I'll keep the thread updated.