dezerving commented 3 days ago

Hello, I need some help with path mapping. Not sure why I can't figure this out but hoping some smart people in here can give me a hand.

I am running Plex on an Ubuntu 22.04 server, my media is located on an Unraid server and the script is running on a Windows server.

Plex database on Ubuntu is at /var/lib/plexmediaserver.... I've exported that SMB share and it is visible at \\<server>\\plexdb\Media on both my Windows machine and a separate Mac machine.

The issue seemingly is at the next point which is the path mapping for the actual media itself.

Plex sees all the media under /mnt/nfs (which is an exported NFS share from Unraid) so for movies it would be /mnt/nfs/movies and so forth. That same Unraid share I have mounted to D:\media as an SMB share on Windows. It is visible. The only difference between the share on Windows and Ubuntu is the root path. I thought I had the correct .env based on my scenario but it doesn't appear so and I can't seem to figure it out.

My .env is as follows:

PLEX_URL=https://192.168.86.3:32400/
PLEX_TOKEN=<token>  
PLEX_BIF_FRAME_INTERVAL=10
THUMBNAIL_QUALITY=4
PLEX_LOCAL_MEDIA_PATH="\\\<server>\plexdb\Media" # this works btw
TMP_FOLDER=C:\Users\<user>\tmp
PLEX_TIMEOUT=60
PLEX_LOCAL_VIDEOS_PATH_MAPPING=D:\media
PLEX_VIDEOS_PATH_MAPPING="/mnt/nfs"
GPU_THREADS=5
CPU_THREADS=5

When running the script here is sample output:

python plex_generate_previews.py
2024/10/16 15:45:38 | ℹ️  - GPU Detection (with AMD Support) was recently added
to this script.
2024/10/16 15:45:38 | ℹ️  - Please log issues here
https://github.com/stevezau/plex_generate_vid_previews/issues
2024/10/16 15:45:38 | ℹ️  - Found NVIDIA GPU
2024/10/16 15:45:38 | ℹ️  - Getting the media files from library 'Movies'
2024/10/16 15:45:59 | ℹ️  - Got 4000 media files for library Movies
⠋ Working... ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━   0% -:--:--    0/40002024/10/16 15:45:59 | ❌  - Skipping as file not found /mnt/nfs/movies/1BR
(2019) {tmdb-611605}/1BR (2019) {tmdb-611605} [Bluray-1080p][DTS 5.1]-iFT.mkv
⠋ Working... ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━   0% -:--:--    0/40002024/10/16 15:45:59 | ❌  - Skipping as file not found /mnt/nfs/movies/2 Fast 2
Furious (2003) {tmdb-584}/2 Fast 2 Furious (2003) {tmdb-584}
[Bluray-1080p][DTS-HD MA 5.1].mkv
2024/10/16 15:45:59 | ❌  - Skipping as file not found /mnt/nfs/movies/2 Guns
(2013) {tmdb-136400}/2 Guns (2013) {tmdb-136400} [Bluray-1080p][AC3 5.1].mkv

Would appreciate any insight on how to resolve this. I have checked all documentation and searched previous issues but can't seem to get an answer to my specific scenario. Thank you in advanced!

stevezau commented 3 days ago

I think you found a bug @dezerving. i just pushed version 1.2.3 which should fix it.

can you test ?

dezerving commented 3 days ago

I think you found a bug @dezerving. i just pushed version 1.2.3 which should fix it.

can you test ?

Hey thanks for that quick work. Looks like that does correctly translate the beginning of the path, but doesn't handle the rest of the path with the slashes (converting / to \). That would probably require a bit more work I assume.

Not sure if there's anything I can do on my end to make Windows paths appear as or accept forward slashes.

PS C:\files\apps\plex_generate_vid_previews> python plex_generate_previews.py
2024/10/16 17:59:48 | ℹ️  - GPU Detection (with AMD Support) was recently added
to this script.
2024/10/16 17:59:48 | ℹ️  - Please log issues here
https://github.com/stevezau/plex_generate_vid_previews/issues
2024/10/16 17:59:48 | ℹ️  - Found NVIDIA GPU
2024/10/16 17:59:48 | ℹ️  - Getting the media files from library 'Movies'
2024/10/16 17:59:58 | ℹ️  - Got 4000 media files for library Movies
⠋ Working... ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━   0% -:--:--    0/40002024/10/16 17:59:58 | ❌  - Skipping as file not found D:\media/movies/2 Fast 2
Furious (2003) {tmdb-584}/2 Fast 2 Furious (2003) {tmdb-584}
[Bluray-1080p][DTS-HD MA 5.1].mkv
2024/10/16 17:59:58 | ❌  - Skipping as file not found D:\media/movies/2 States
(2014) {tmdb-259720}/2 States (2014) {tmdb-259720} [Bluray-1080p][DTS 5.1].mkv
2024/10/16 17:59:58 | ❌  - Skipping as file not found D:\media/movies/8 Found
Dead (2023) {tmdb-1035705}/8 Found Dead (2023) {tmdb-1035705} [Bluray-1080p][AC3
5.1]-OFT.mkv

dezerving commented 3 days ago

Just doing a quick look, it looks like we could use the pathlib module so the paths are correctly handled depending on OS.

https://medium.com/@ageitgey/python-3-quick-tip-the-easy-way-to-deal-with-file-paths-on-windows-mac-and-linux-11a072b58d5f

stevezau commented 3 days ago

@dezerving you are right.. I don't handle windows too well. Let's fix that..

Just confirming.. you are running the script directly right? not in docker?

dezerving commented 3 days ago

@dezerving you are right.. I don't handle windows too well. Let's fix that..

Just confirming.. you are running the script directly right? not in docker?

Yes directly on Windows, not Docker.

Docker on Windows (through WSL) was having issues as well - although with your fix in 1.2.3 I could probably try it again (since I can use forward slashes on there). Will give it a go later to see if that works.

Regardless yeah it'd be good to fix the paths from the ground up so there's no issues in the future. This tool is gonna be so useful for my 50TB library - had to redo because my db got corrupted.

stevezau commented 2 days ago

Ill sort this out in the next day or two.

dezerving commented 2 days ago

Ill sort this out in the next day or two.

Sounds good, looking forward to that update. Will help test as needed.

dezerving commented 2 days ago

In the meantime I was able to get this script to run with my setup using Docker (since you can use forward and backward slashes freely). I'll leave the documentation below (or PR if you want):

Guide for Docker script, Ubuntu Plex, and Unraid (or any NAS) media location

Note: this does not require you to change any paths within Plex itself (as opposed to the Unraid guide within the docs currently). I'll explain all the path mapping and volumes below.

Assumptions:

Media file location is on a NAS (Unraid for me)
- File path for media files for this guide: /mnt/user/data/media
  - You have exported this as an SMB share and NFS share (NFS for linux and SMB for Windows)
    - SMB available at \\<server or IP>\\data\media on Windows
    - NFS shares available at /<server or IP>:/mnt/user/data/media/<folder> mounted at /mnt/nfs/<folder> within Ubuntu
      - Example: /<server or IP>:/mnt/user/data/media/movies is mounted at /mnt/nfs/movies
Plex server is on linux-based system (Ubuntu for me)
- Your media files path (path that Plex sees) is: /mnt/nfs (if different change accordingly)
  - Example file path in Plex: /mnt/nfs/movies/1BR (2019)/1BR (2019).mkv
- You have exported your Plex database folder as an SMB share (/var/lib/plexmediaserver...)
  - My snippet for the share in smb.conf file as reference:
```
[plexdb]
path = "/var/lib/plexmediaserver/Library/Application Support/Plex Media Server"
browsable = yes
writable = yes
read only = no
```
Script is being run on Docker (on Windows)
- Use the .env file below as reference
- Set PLEX_LOCAL_MEDIA_PATH to the exported Plex database folder
  - I mapped plexdb to T: drive on Windows
  - Making the final path to the folder with the localhost folder in it as: T:\Media
- At the bottom of the .env file I added volumes section where I defined the share from Unraid as type cifs, pretty self-explanatory
  - Then mounted that as a volume inside the docker container to /mnt/nfs
- I also altered the way the GPU is accessed (copied from Immich I think) but it worked as the original way was giving me errors on Windows

.env file:

---
services:
  previews:
    image: stevezzau/plex_generate_vid_previews:latest
    container_name: plex-thumbs
    environment:
      - PLEX_URL=https://<plex_server_ip>:32400 
      - PLEX_TOKEN=<token>
      - PLEX_BIF_FRAME_INTERVAL=10
      - THUMBNAIL_QUALITY=4
      - PLEX_LOCAL_MEDIA_PATH=/media                # be sure this is the correct volume mapped from below
      - TMP_FOLDER=/tmp/previews                            # probably want to make this RAM disk in the future
      - PLEX_TIMEOUT=60
      - GPU_THREADS=5
      - CPU_THREADS=5
    volumes:
      - T:/Media:/media                                                 # volume of your plex db folder (SMB mapped to network drive)
      - plexmedia:/mnt/nfs                                          # CIFS mount of your actual media mapped at bottom of file
    deploy:
      resources:
        reservations:
          devices:
            - driver: nvidia
              count: 1
              capabilities:
                - gpu

volumes:
  plexmedia:
    driver_opts:
      type: cifs
      device: "//<server or IP>/data/media"
      o: username=user,password=password

Will reformat later but this is the gist.

stevezau commented 2 days ago

@dezerving can you test the script on windows again? I just pushed a change that will hopefully fix it https://github.com/stevezau/plex_generate_vid_previews/commit/b374f8bb21eebbbef22413a82d4806db921a6ba0

note, i haven't pushed a new docker release yet, i will oce you confirm the above works.

stevezau commented 2 days ago

Re the Docs, this is great, perhaps you can look at the existing section in the README and update it as required via a PR?

dezerving commented 2 days ago

@dezerving can you test the script on windows again? I just pushed a change that will hopefully fix it https://github.com/stevezau/plex_generate_vid_previews/commit/b374f8bb21eebbbef22413a82d4806db921a6ba0

note, i haven't pushed a new docker release yet, i will oce you confirm the above works.

Will test early tomorrow morning and report back.

dezerving commented 2 days ago

Re the Docs, this is great, perhaps you can look at the existing section in the README and update it as required via a PR?

Will also look at cleaning up docs over the next week. Maybe would require more time if implementing anything from #53.

dezerving commented 1 day ago

@dezerving can you test the script on windows again? I just pushed a change that will hopefully fix it b374f8b

note, i haven't pushed a new docker release yet, i will oce you confirm the above works.

So I just tested and it appears like the slashes do get corrected to the correct style but it still cannot find my files (even though they exist).

PS C:\files\apps\plex_generate_vid_previews> python plex_generate_previews.py
2024/10/18 11:30:27 | ℹ️  - GPU Detection (with AMD Support) was recently added
to this script.
2024/10/18 11:30:27 | ℹ️  - Please log issues here
https://github.com/stevezau/plex_generate_vid_previews/issues
2024/10/18 11:30:27 | ℹ️  - Found NVIDIA GPU
2024/10/18 11:30:27 | ℹ️  - Getting the media files from library 'Movies'
2024/10/18 11:30:37 | ℹ️  - Got 4004 media files for library Movies
⠋ Working... ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━   0% -:--:--    0/40042024/10/18 11:30:37 | ❌  - Skipping as file not found D:\media\movies\1BR
(2019) {tmdb-611605}\1BR (2019) {tmdb-611605} [Bluray-1080p][DTS 5.1]-iFT.mkv
2024/10/18 11:30:37 | ❌  - Skipping as file not found D:\media\movies\2 Fast 2
Furious (2003) {tmdb-584}\2 Fast 2 Furious (2003) {tmdb-584}
[Bluray-1080p][DTS-HD MA 5.1].mkv

When I paste D:\media\movies\1BR (2019) {tmdb-611605} into Explorer just to make sure I'm not being dumb... I see the file correctly existing exactly as titled in the logs.

Not sure if this is an issue with the path mapping implementation or if I'm screwing up the paths somehow. I've tried quotes, non-quotes, etc. Nothing has allowed the script to actually see the files correctly directly on Windows (works through Docker WSL).

My .env:

PLEX_URL=https://<plex_server>:32400/
PLEX_TOKEN=<token>  
PLEX_BIF_FRAME_INTERVAL=10
THUMBNAIL_QUALITY=4
PLEX_LOCAL_MEDIA_PATH="\\\<server>\plexdb\Media"
TMP_FOLDER=C:\Users\<user>\tmp
PLEX_TIMEOUT=60
PLEX_LOCAL_VIDEOS_PATH_MAPPING=D:\media
PLEX_VIDEOS_PATH_MAPPING=/mnt/nfs
GPU_THREADS=1
CPU_THREADS=0

stevezau commented 1 day ago

@dezerving hmm that's odd. Can you do a quick test..

open python3 in your console and paste in the below.

import os
path = 'D:\media\movies\2 Fast 2 Furious (2003) {tmdb-584}\2 Fast 2 Furious (2003) {tmdb-584} [Bluray-1080p][DTS-HD MA 5.1].mkv'
print(os.path.exists(path))
print(os.path.isfile(path))

What's the output?

dezerving commented 1 day ago

@dezerving hmm that's odd. Can you do a quick test..

open python3 in your console and paste in the below.
import os
path = 'D:\media\movies\2 Fast 2 Furious (2003) {tmdb-584}\2 Fast 2 Furious (2003) {tmdb-584} [Bluray-1080p][DTS-HD MA 5.1].mkv'
print(os.path.exists(path))
print(os.path.isfile(path))
What's the output?

Changed the movie since that movie was recently upgraded and that's not the actual movie file path. The movie (1BR) is actually present at this path: D:\media\movies\1BR (2019) {tmdb-611605}\1BR (2019) {tmdb-611605} [Bluray-1080p][DTS 5.1][x264]-iFT.mkv.

>>> import os
>>> path = 'D:\media\movies\1BR (2019) {tmdb-611605}\1BR (2019) {tmdb-611605} [Bluray-1080p][DTS 5.1][x264]-iFT.mkv'
>>> print(os.path.exists(path))
False
>>> print(os.path.isfile(path))
False
>>> print(path)
D:\media\moviesBR (2019) {tmdb-611605}BR (2019) {tmdb-611605} [Bluray-1080p][DTS 5.1][x264]-iFT.mkv

Looks like Python cannot see that path on Windows because the single backslashes are causing issues and are being lost. I would think you have to double escape every backward slash but that's not the case either as the first two backslashes are fine. It's only the backslashes that are directly before a number that have to be double escaped. Even then, that doesn't result in the path being seen.

Going a bit simpler, I just wanted Python to be able to see a directory on the C:\ drive. I was successful in seeing that directory.

>>> path = 'C:\files'
>>> print(path)
C:
iles
>>> print(os.path.exists(path))
False
>>> path = 'C:\\files'
>>> print(path)
C:\files
>>> print(os.path.exists(path))
True
>>> path = 'C:/files'
>>> print(path)
C:/files
>>> print(os.path.exists(path))
True

So both ways work for files on the C:\ drive but for some reason do not work for files on the D:\ drive (network mapped drive).

>>> path = 'D:/media/movies/1BR (2019) {tmdb-611605}/1BR (2019) {tmdb-611605} [B
luray-1080p][DTS 5.1][x264]-iFT.mkv'
>>> print(os.path.exists(path))
False
>>> print(os.path.isfile(path))
False
>>> print(os.path.isfile(path))

It's so annoying but this is how far I've gotten so far. Will keep trying things and update the post. But let me know if you find an answer before I do lol.

stevezau commented 1 day ago

@dezerving i'm not sure what is going on but i don't have a windows machine to test it.. If you find out what is happening ill update the code but without access to your machine it is a bit difficult as i cannot reproduce.

However... can you try run all those again but use double quotes not single quotes and share the output?

dezerving commented 1 day ago

Works on Windows for me with:

PLEX_LOCAL_VIDEOS_PATH_MAPPING="\\\<server>/data/media"

Tried answers from here after a bunch of other places and this worked. Definitely not solved but workable for my case right now.

How I came to the solution above:

>>> path = 'D:/media/movies/1BR (2019) {tmdb-611605}'
>>> print(os.path.exists(path))
False
>>> print(path)
D:/media/movies/1BR (2019) {tmdb-611605}
>>> path = '\\\<server>/data/media'
>>> print(path)
\\<server>/data/media
>>> print(os.path.exists(path))
True

Script runs fine directly on Windows now:

2024/10/18 17:56:19 | ℹ️  - GPU Detection (with AMD Support) was recently added to this script.
2024/10/18 17:56:19 | ℹ️  - Please log issues here https://github.com/stevezau/plex_generate_vid_previews/issues
2024/10/18 17:56:19 | ℹ️  - Found NVIDIA GPU
2024/10/18 17:56:19 | ℹ️  - Getting the media files from library 'Movies'
2024/10/18 17:56:29 | ℹ️  - Got 4005 media files for library Movies
⠋ Working... ━━━━━━━╸━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━  20% 0:02:32  798/40052024/10/18 17:58:32 | ℹ️  - Generated Video Preview for \\<server>\data\media\movies\The Convert (2024) {tmdb-1066262}\The
Convert (2024) {tmdb-1066262} [Bluray-1080p][AC3 5.1]-BHDStudio.mp4 HW=True TIME=85.3seconds SPEED=85.8x

dezerving commented 1 day ago

@dezerving i'm not sure what is going on but i don't have a windows machine to test it.. If you find out what is happening ill update the code but without access to your machine it is a bit difficult as i cannot reproduce.

However... can you try run all those again but use double quotes not single quotes and share the output?

Double quotes or single quotes did not change anything for me. You still lose the backslash when a number immediately follows it.

Looks to be an issue with how Python sees network drives? Because it can see local drives fine with any of the methods to convert paths (double backslash \\ or just converting \ to / directly).

Output:

>>> path = "D:\media\movies\1BR (2019) {tmdb-611605}\1BR (2019) {tmdb-611605} [Bluray-1080p][DTS 5.1][x264]-iFT.mkv"
>>> print(path)
D:\media\moviesBR (2019) {tmdb-611605}BR (2019) {tmdb-611605} [Bluray-1080p][DTS 5.1][x264]-iFT.mkv
>>> print(os.path.exists(path))
False

stevezau commented 22 hours ago

Hey @dezerving, hmm looking at your last output It looks like the print(path) is incorrect tho?

It shows as D:\media\moviesBR (2019) {tmdb-611605}BR

when it should be D:\media\moviesBR (2019) {tmdb-611605}\1BR

Something odd is going on.

can you try with raw string mode?

path = r"D:\media\movies\1BR (2019) {tmdb-611605}\1BR (2019) {tmdb-611605} [Bluray-1080p][DTS 5.1][x264]-iFT.mkv"

can you also try os.stat(path) on the path? and share the output..

dezerving commented 12 hours ago

Hey @dezerving, hmm looking at your last output It looks like the print(path) is incorrect tho?

It shows as D:\media\moviesBR (2019) {tmdb-611605}BR

when it should be D:\media\moviesBR (2019) {tmdb-611605}\1BR

Something odd is going on.

It actually should be D:\media\movies\1BR (2019) {tmdb-611605}\1BR. My suspicion is it takes \{any number} and just removes it from the path completely.

can you try with raw string mode?

can you also try os.stat(path) on the path? and share the output..

Sure. And this is where it gets interesting. It's able to actually get the correct path but Python is still unable to see it. I'm thinking this is because of it being a network drive that is mapped to a letter (D:\).

Using the UNC path as shown in my previous comment does work for me.

Raw string mode:

>>> path = r"D:\media\movies\1BR (2019) {tmdb-611605}\1BR (2019) {tmdb-611605} [Bluray-1080p][DTS 5.1][x264]-iFT.mkv"
>>> print(path)
D:\media\movies\1BR (2019) {tmdb-611605}\1BR (2019) {tmdb-611605} [Bluray-1080p][DTS 5.1][x264]-iFT.mkv
>>> print(os.path.exists(path))
False
>>> print(os.stat(path))
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
FileNotFoundError: [WinError 3] The system cannot find the path specified: 'D:\\media\\movies\\1BR (2019) {tmdb-611605}\\1BR (2019) {tmdb-611605} [Bluray-1080p][DTS 5.1][x264]-iFT.mkv'

I think this thread will be of use to anyone who has a "complicated" setup like I do.

stevezau commented 4 hours ago

Ok got it, thanks @dezerving

I pushed https://github.com/stevezau/plex_generate_vid_previews/commit/22e26308f2b2fa47de094a05babcffb401f8ebf5 to add an FAQ.

Closing this one out now.

stevezau / plex_generate_vid_previews

Path mapping help needed (Plex on Ubuntu, Script on Windows, Media on Unraid) #52

Guide for Docker script, Ubuntu Plex, and Unraid (or any NAS) media location

Assumptions:

.env file: