Closed nbn22385 closed 9 months ago
I-am-PUID-0
commented
9 months ago Thanks for the very detailed response.
A couple of notes:
PLEX_MOUNT_DIR likely needs to be changed to /torrents[plex] refreshing movie library section/s: "MoviesRD" <---- Attemping to refresh Plex is from plex_debrid. If using PLEX_REFRESH, you may want to disable the Library update services for Plex from within plex_debrid.
nbn22385
commented
9 months ago A follow-up: I changed PLEX_MOUNT_DIR=/torrents and did the following tests,
First I set PLEX_REFRESH=false and used the plex_debrid Plex Library Update Service, the Overseerr request goes through and Plex updates successfully.
Then I set PLEX_REFRESH=true and removed Plex from "Library Update Services" in the plex_debrid menu and tried the following:
I-am-PUID-0
commented
9 months ago I assume the environment variables that =SECRET are actually uncommented when running?
Do you ever see anything like the below?
Jan 13, 2024 12:36:39 - DEBUG - Zurg w/ RealDebrid subprocess: manager Output of hook on_library_update:
Jan 13, 2024 12:36:39 - INFO - Zurg w/ RealDebrid subprocess: Starting Plex Update for __all__/Reacher.S02E07.MULTI.2160p.WEB-DL.HDR.H265-AOC
Jan 13, 2024 12:36:39 - INFO - Zurg w/ RealDebrid subprocess: Starting Plex Update for shows/Reacher.S02E07.MULTI.2160p.WEB-DL.HDR.H265-AOCI assume the environment variables that
=SECRETare actually uncommented when running?
They are commented out since I followed the secrets section of the Readme and have text files for each of the secrets along with the secrets: keyword in the config.
Do you ever see anything like the below?
Nope, after another attempt the closest I got was this (PLEX_REFRESH=true with Plex Library Update Services removed from plex_debrid:
pd_zurg | Jan 13, 2024 21:45:39 - INFO - Zurg w/ RealDebrid subprocess: manager Detected changes! Refreshing 14 torrents
pd_zurg | Jan 13, 2024 21:45:39 - DEBUG - Zurg w/ RealDebrid subprocess: realdebrid Got info for torrent SRRVQEIAYVZNK (progress=100%)
pd_zurg | Jan 13, 2024 21:45:39 - DEBUG - Zurg w/ RealDebrid subprocess: manager Saved torrent SRRVQEIAYVZNK to file
pd_zurg | Jan 13, 2024 21:45:39 - DEBUG - Zurg w/ RealDebrid subprocess: manager Fetched info for 14 torrents
pd_zurg | Jan 13, 2024 21:45:39 - INFO - Zurg w/ RealDebrid subprocess: manager Compiled into 14 torrents, 0 were missing info
pd_zurg | Jan 13, 2024 21:45:39 - DEBUG - Zurg w/ RealDebrid subprocess: manager Added Indiana.Jones.And.The.Dial.Of.Destiny.2023.1080p.REMUX.ENG.ITA.LATINO.DTS-HD.Master.DDP5.1.MKV-BEN.THE.MEN to [movies]
pd_zurg | Jan 13, 2024 21:45:39 - INFO - Zurg w/ RealDebrid subprocess: manager Finished refreshing torrents
pd_zurg | Jan 13, 2024 21:45:39 - DEBUG - Zurg w/ RealDebrid subprocess: manager Triggering hook on_library_update for 2 path(s)
pd_zurg | Jan 13, 2024 21:45:39 - DEBUG - Zurg w/ RealDebrid subprocess: manager Found 0 torrents to repair
pd_zurg | Jan 13, 2024 21:45:39 - DEBUG - Zurg w/ RealDebrid subprocess: manager Finished repairing all torrentsDo you have a working config available for viewing?
I-am-PUID-0
commented
9 months ago Right. Sorry. Working from my phone at the moment.
The Triggering hook on_library_update for 2 path(s) is the zurg call to the plex_refresh.py file in /zurg/RD. Unfortunately, since the zurg app is calling the update script, any logging is limted by zurg until manager Output of hook on_library_update: is displayed.
Can you check your config.yml in the /zurg/RD directory for the following:
on_library_update: |
tmpfile=$(mktemp)
for arg in "$@"
do
echo "$arg" >> "$tmpfile"
done
unique_args=$(sort -u "$tmpfile")
if [ -n "$unique_args" ]; then
IFS=$'\n'
for line in $unique_args; do
python plex_refresh.py "$line"
done
unset IFS
fi
rm "$tmpfile"
# network configsYes my /pd_zurg/RD/config.yml has that code. I wonder if I have a path issue/mismatch somewhere. Do my volumes look ok?
pd_zurg:
...
volumes:
- /pd_zurg/config:/config
- /pd_zurg/log:/log
- /pd_zurg/cache:/cache
- /pd_zurg/RD:/zurg/RD
- /pd_zurg/mnt:/data:shared
plex:
...
volumes:
...
- /pd_zurg/mnt/pd_zurg/:/torrents
I also noticed plex_debrid doesn't like when I have no library update service set up:
Looks like your current settings didnt pass preflight checks. Please edit the following setting/s:
Library update services: Please add at least one libary update service.Thanks for all the help.
nbn22385
commented
9 months ago Update! After much debugging, I've figured out the issue is related to my use of Docker secrets. When I revert to placing my api keys in the docker environment: section, the Plex update works across the board for files added both through Overseerr and DebridMediaManager (Produces the INFO - Zurg w/ RealDebrid subprocess: Starting Plex Update for ... log statements you had asked about).
That answers my original question regarding the ability to update Plex with manually added files 👍
The issue remains why my Docker secrets setup is not working or producing any error logs, but I will try regenerating their text files and see what I can find out. I had a feeling it had something to do with creating my secret files with printf/echo -n vs echo, but haven't been able to prove it yet. I put a print statement into the main() of plex_refresh.py script and it seems like the script is not called at all when I utilize docker secrets.
Are these variables getting set properly in the docker secrets use case, or should they be using the app variables PLEXADD/PLEXTOKEN instead?
I-am-PUID-0
commented
9 months ago Thanks for continuing to troubleshoot! I believe you may be correct in your assessment. Since the Zurg application directly calls plex_refresh.py, it may not be in the same namespace as main.py. I'll see if I can resolve this by importing the base init.py to the plex_refresh.py. I'll report back once I've confirmed and resolved the issue.
I-am-PUID-0
commented
9 months ago I won't have time to test it for a bit, but if you'd like, you can try modifying the plex_refresh.py in the /zurg/RD directory with the below:
import sys
sys.path.append('../')
from base import PLEXTOKEN, PLEXADD, PLEXMOUNT, RCLONEMN
from plexapi.server import PlexServer
import os
import time
# Configuration
plex_url = PLEXADD.replace("'", "").replace('"', '')
token = PLEXTOKEN.replace("'", "").replace('"', '')
plex_mount = PLEXMOUNT.replace("'", "").replace('"', '')
zurg_mount = f"/data/{RCLONEMN}"So far I couldn't get it to work with those modifications in both /zurg/RD/plex_refresh.py and /zurg/plex_refresh.py inside the container since I wasn't sure which was the true version. I also tried hardcoding sys.path.append('/base') since base seems to be 2 levels up from /zurg/RD/plex_refresh.py. Perhaps I am testing it wrong and you will have better results.
I-am-PUID-0
commented
9 months ago So far I couldn't get it to work with those modifications in both
/zurg/RD/plex_refresh.pyand/zurg/plex_refresh.pyinside the container since I wasn't sure which was the true version. I also tried hardcodingsys.path.append('/base')since base seems to be 2 levels up from/zurg/RD/plex_refresh.py. Perhaps I am testing it wrong and you will have better results.
Thanks. I believe it may not have been going far enough for the parent directories.
the /zurg/RD/plex_refresh.py is the one that will be called by Zurg when new content is added. /zurg/plex_refresh.py is the default, which gets added to the subdirectories if the file does not exist.
Please try the below:
import os
import sys
import time
from plexapi.server import PlexServer
from pathlib import Path
plex_refresh_path = Path(__file__).resolve()
project_path = plex_refresh_path.parent.parent.parent
sys.path.append(str(project_path))
from base import PLEXTOKEN, PLEXADD, PLEXMOUNT, RCLONEMN
# Configuration
plex_url = PLEXADD.replace("'", "").replace('"', '')
token = PLEXTOKEN.replace("'", "").replace('"', '')
plex_mount = PLEXMOUNT.replace("'", "").replace('"', '')
zurg_mount = f"/data/{RCLONEMN}"
zurg_timeout = 300 # 5 minutes in seconds for Zurg file availability
plex_timeout = 60 # Maximum time to wait for Plex to process the refresh
wait_increment = 1 # Time increment for each wait step
max_retries = 20 # Maximum number of retries for Plex refresh
### Do not alter below ###I updated to your latest suggestion and everything seems to work when using the Docker secrets 👍. The Plex library was updated in all of my scenarios: Overseerr request, Plex watchlist, and DebridMediaManager.
Just one final question if you have time: do you have a certain dependency order between your plex and pd_zurg services? I didn't see this documented anywhere but was wondering if it was possible to avoid this error on startup:
pd_zurg | [18/01/24 12:12:44] [plex error]: couldnt reach local plex server at: http://ubuntu-server.local:32400 to determine library sections. Make sure the address is correct, the server is running, and youve set up at least one library.I typically start pd_zurg and then plex since the rclone mount needs to be present for Plex to start up, but I'm not sure the above error can be avoided on initial startup. It might be a non-issue since the services seem to work fine once Plex is up a few seconds later. Using depends_on seems to start Plex too soon before the mount is available.
I ended up going with this method to utilize your healthcheck, although there is a delay of about 30 seconds before Plex is up (it's much faster if I manually start the services). Maybe that's just how long the healthcheck takes.
plex:
depends_on:
pd_zurg:
condition: service_healthy$ time docker compose up -d plex
[+] Running 2/2
✔ Container pd_zurg Healthy 0.0s
✔ Container plex Started 0.0s
docker compose up -d plex 0.36s user 0.22s system 0% cpu 1:01.00 totalEdit: I kind of answered my own question about the long delay (it depends on the --interval of the healthcheck); I overrode your healthcheck interval from 60s to 10s to reduce my startup time and so far it's working ok.
pd_zurg:
healthcheck:
interval: 10sI updated to your latest suggestion and everything seems to work when using the Docker secrets 👍. The Plex library was updated in all of my scenarios: Overseerr request, Plex watchlist, and DebridMediaManager.
Just one final question if you have time: do you have a certain dependency order between your
plexandpd_zurgservices? I didn't see this documented anywhere but was wondering if it was possible to avoid this error on startup:pd_zurg | [18/01/24 12:12:44] [plex error]: couldnt reach local plex server at: http://ubuntu-server.local:32400 to determine library sections. Make sure the address is correct, the server is running, and youve set up at least one library.I typically start
pd_zurgand thenplexsince the rclone mount needs to be present for Plex to start up, but I'm not sure the above error can be avoided on initial startup. It might be a non-issue since the services seem to work fine once Plex is up a few seconds later. Usingdepends_onseems to start Plex too soon before the mount is available.I ended up going with this method to utilize your
healthcheck, although there is a delay of about 30 seconds before Plex is up (it's much faster if I manually start the services). Maybe that's just how long the healthcheck takes.plex: depends_on: pd_zurg: condition: service_healthy$ time docker compose up -d plex [+] Running 2/2 ✔ Container pd_zurg Healthy 0.0s ✔ Container plex Started 0.0s docker compose up -d plex 0.36s user 0.22s system 0% cpu 1:01.00 totalEdit: I kind of answered my own question about the long delay (it depends on the
--intervalof the healthcheck); I overrode your healthcheck interval from 60s to 10s to reduce my startup time and so far it's working ok.pd_zurg: healthcheck: interval: 10s
Excellent! Glad to see it is working. Thank you for testing. I'll add the fix to the next release. I also like the work around to speed up deployment. I'll add it to the wiki as well.
I-am-PUID-0
commented
9 months ago Resolved in Version [2.0.2]
Is there a way to configure pd_zurg to refresh my Plex library when files are added by a means other than requests/watchlist, but simply when changes are detected? This could come down to a configuration issue I missed.
Scenario 1: When I add a file to my RD library manually (via DebridMediaManager), pd_zurg detects the change to the rclone mount but the Plex library does not refresh automatically. I need to "Scan Library Files" in Plex to see the new file. This is also true for files I delete manually.
Scenario 2: When I request a file via Overseerr, the file is added to my RD library and pd_zurg (attempted to) refresh the Plex library automatically. However, the file is not actually detected by Plex until I manually update, because from the logs below, it looks like pd_zurg is sending the update command with the wrong path. This looks a lot like the nested movie folder issue reported in #4. (Plex is attempting to process
/torrents/movies/MOVIE.mkvrather than/torrents/movies/MOVIE/and throws an error). Not sure why Plex is getting that bad path anyway and not just refreshing the library section as a whole.pd_zurg logs:
Plex logs:
^ That path in the last line is not valid, the movie file resides in a folder with the same name as the file.
My configuration is here.