search

mynttt / UpdateTool

A tool to update the IMDB ratings for Plex libraries that contain movies/series and use the IMDB agent to receive ratings
GNU General Public License v3.0
257 stars 12 forks source link

DSM 7 and new plex location #66

Closed mrjohnpoz closed 3 years ago

mrjohnpoz commented 3 years ago

Now that plex running on DSM 7 has moved the folder - how to configure Updatetool? Will it work even?

plex/library/application support/Plex Media Server

Is not longer there - and were the plex server moved is not visable to docker..

mynttt commented 3 years ago

They probably have moved the folder into a volume or something like that. Could you access the shell of the container and run this command?

find / -type d -name "Plex Media Server"

It might take some time but it should output wherever the folder is now. Then we can look into finding a way on how to support it.

mrjohnpoz commented 3 years ago

Looks like they moved the db to here /volume1/@appstore/PlexMediaServer/Resources

Your looking for the com.plexapp.plugins.library.db right? That is that dir..

mynttt commented 3 years ago

Can you give me the output of the tree -d -L 5 command in the PlexMediaServer?

mynttt commented 3 years ago

Ahh havent read the last part. Wait I'll look how UpdateTool determaines the DIR and then I'll push some fix.

mynttt commented 3 years ago

You can now override the standard location with the pre-given directory structure by using the environment variable OVERRIDE_DATABASE_LOCATION. So in your case OVERRIDE_DATABASE_LOCATION="/volume1/@appstore/PlexMediaServer/Resources".

mrjohnpoz commented 3 years ago

That doesn't seem to work - still not starting

Start container confident_perlman failed: {"message":"Bind mount failed: '/volume1/Plex/Library/Application Support/Plex Media Server' does not exists"}.

How to point the /plexdata folder? There is no way to point there in the docker gui on synology - guess could try recreating with cmdline - but if I set plexdata - what is the point of the override variable?

mynttt commented 3 years ago

Is it possible to remove the plexdata binding? It will not be required anymore as your override value is used inside the container and will point on the path that you set there.

mrjohnpoz commented 3 years ago

Yeah I did that - but then it gives another error about config folder

java.sql.SQLException: path to '/config/"/volume1/@appstore/PlexMediaServer/Resources"/com.plexapp.plugins.library.db': '/config/"' does not exist

mynttt commented 3 years ago

That's a strange path. I guess the "" are not needed as the variable is being escaped by the GUI already.

mrjohnpoz commented 3 years ago

I tried that as well.. let me try again

mrjohnpoz commented 3 years ago

Now get this when not using "

java.sql.SQLException: path to '/volume1/@appstore/PlexMediaServer/Resources/com.plexapp.plugins.library.db': '/volume1' does not exist

mynttt commented 3 years ago

Is it possible for you to enter the shell of the container and run tree -d -L 2 on / and post the output here?

mynttt commented 3 years ago

It's strange that volume1 is supposed to not exist when the folder is mounted under '/volume1/@appstore/PlexMediaServer/Resources'. Or is that the path outside of the container? The tool will only have access to paths inside of the container.

mynttt commented 3 years ago

If it is outside just mount '/volume1/@appstore/PlexMediaServer/Resources' to /plexdata and set the override to /plexdata.

Without the override the tool will search for /plexdata + Plugin Support/Database (which will not work since the DB is directly in /Resources)

mrjohnpoz commented 3 years ago

plex is not a docker, plex is native plex app running on the synology. The new dsm7 package moved the path from just a shared folder plex to this location.

mrjohnpoz commented 3 years ago

You can not mount '/volume1/@appstore/PlexMediaServer/Resources' in the docker gui as a volume.. Have to try with cmd line control of docker

mynttt commented 3 years ago

It's not possible to mount it because the GUI filters that location? I never used anything from synology so I have no idea how much control you have. Could you maybe create a symlink (soft) from a place where the GUI allows you to create a bind mount to /volume1/@appstore/PlexMediaServer/Resources? And then mount the symlink as /plexdata?

mrjohnpoz commented 3 years ago

Ok I created a symlink to the folder /plexdata.. But now its saying no libraries exist in the db.


[INFO ] - 2021-07-17 00:22:27 @ Main.loadImplementation: Loaded implementation << imdb-docker << updatetool.imdb.ImdbDockerImplementation
[INFO ] - 2021-07-17 00:22:27 @ Main.main: Running version: 1.5.8
[INFO ] - 2021-07-17 00:22:27 @ Main.main: Args: {imdb-docker={schedule=12}}
[INFO ] - 2021-07-17 00:22:27 @ Main.main: << INIT: imdb-docker @ {schedule=12} >>
[INFO ] - 2021-07-17 00:22:28 @ TmdbApiV3.<init>: Testing TMDB API (v3) key: 0e0fab<snipped>
[INFO ] - 2021-07-17 00:22:29 @ TmdbApiV3.<init>: Test passed. API Key is valid.
[INFO ] - 2021-07-17 00:22:29 @ ImdbDockerImplementation.bootstrap: TMDB API key enabled TMDB <=> IMDB matching. Will process TMDB backed Movie and TV Series libraries and TMDB orphans.
[INFO ] - 2021-07-17 00:22:29 @ TvdbApiV3.<init>: Testing TVDB API (v3) authorization apikey: 504f7bed6<snipped>
[INFO ] - 2021-07-17 00:22:29 @ TvdbApiV3.<init>: Test passed. API Key is valid.
[INFO ] - 2021-07-17 00:22:29 @ ImdbDockerImplementation.bootstrap: TVDB API (TVDB_V3) authorization enabled IMDB rating update for Movies and TV Series with the TVDB agent.
[INFO ] - 2021-07-17 00:22:29 @ ImdbDockerImplementation.bootstrap: Starting IMDB Watchdog
[INFO ] - 2021-07-17 00:22:29 @ ImdbDockerImplementation.bootstrap: Plex data dir: /plexdata
[INFO ] - 2021-07-17 00:22:29 @ ImdbDockerImplementation.bootstrap: Ignoring library with ID: 8
[INFO ] - 2021-07-17 00:22:29 @ ImdbDockerImplementation.bootstrap: Ignoring library with ID: 12
[INFO ] - 2021-07-17 00:22:29 @ ImdbDockerImplementation.bootstrap: Unlocking library for IMDB TV Show rating update for the new TV Show agent: 6
[INFO ] - 2021-07-17 00:22:29 @ ImdbDockerImplementation.bootstrap: Capabilities: [TMDB, TVDB, VERBOSE_XML_ERROR_LOG]
[INFO ] - 2021-07-17 00:22:29 @ ImdbDockerImplementation.getDatabaseLocation: Custom path specified @ /plexdata
[INFO ] - 2021-07-17 00:22:29 @ Main.main: << INIT SUCCESS >>
[INFO ] - 2021-07-17 00:22:29 @ Scheduler.go: Scheduler is loading tasks... Blocking until completely set-up and ready to go.
[INFO ] - 2021-07-17 00:22:29 @ Scheduler.go: Scheduling tasks...
[INFO ] - 2021-07-17 00:22:29 @ Scheduler.lambda$go$0: Scheduled imdb-docker task to run @ every 12 hour(s).
[INFO ] - 2021-07-17 00:22:29 @ Scheduler.lambda$go$1: Queued task imdb-docker for immediate execution.
[INFO ] - 2021-07-17 00:22:29 @ Scheduler.go: Running supplied tasks immediately NOW!
[INFO ] - 2021-07-17 00:22:29 @ TaskWrapper.run: ================================================
[INFO ] - 2021-07-17 00:22:29 @ TaskWrapper.run: Starting task: imdb-docker | Execution count: 0
[INFO ] - 2021-07-17 00:22:29 @ TaskWrapper.run: ================================================
[ERROR] - 2021-07-17 00:22:30 @ ImdbDockerImplementation$ImdbBatchJob.run: FATAL ERROR :: NO LIBRARIES EXIST IN DATABASE. EXITING NOW.

My guess would be the package for dsm7 changes a bunch of stuff.. I can send you the current db if you like..

mynttt commented 3 years ago

Probably a good idea. Was there any migration process for you? Sounds like it's a fresh Plex installation without any libraries in that log.

mrjohnpoz commented 3 years ago

Yeah there was a migration when you installed it - took quite some time as well.. I don't think the time was in anything major to do with the db, but the moving of all the metadata files, etc. So they prob had to change some paths and such in the db. I just emailed copy of database from my dsm7... plex.. Version 1.23.5.4801

All of my data is there.. but maybe they renamed some stuff or redid some structure?

mynttt commented 3 years ago

The database is fine and contains all the libraries. It also works on my system. There has to be some issue with the symlink. A database does exist the question is only if it is the same then. Because if the database would not exist there would be an exception with the queries. What happens when you run a ls -ll *.db on the symlink?

mynttt commented 3 years ago

How did you get the backup tho? It had a really strange name?

mrjohnpoz commented 3 years ago

Got the backup from the troubleshooting tab in plex.. 

ash-4.4# pwd
/volume1/docker/plexdata
ash-4.4# ls -ll *.db
-rw-rw-r-- 1 PlexMediaServer PlexMediaServer 318464 Jul  9 15:45 com.plexapp.plugins.library.db
ash-4.4#

Does the docker maybe not have permission?

mynttt commented 3 years ago

Wait did you create a symlink of the database file or of the folder?

mynttt commented 3 years ago

There's no way the databases are the same if it's of the folder. The database in the ls command is tiny in comparison to what I've received per email. Can you execute the same ls command in the Resource folder?

mrjohnpoz commented 3 years ago

I created the symlink of the folder..

here is the of the resource folder

ash-4.4# pwd
/volume1/@appstore/PlexMediaServer/Resources
ash-4.4# ls -ll *.db
-rw-rw-r-- 1 PlexMediaServer PlexMediaServer 318464 Jul  9 15:45 com.plexapp.plugins.library.db
ash-4.4#
mynttt commented 3 years ago

The database is not the one that Plex uses. It has a size of 318KB which would mean it's probably empty and only contains the schema and some default values.

Can you run this on your system? find / -name "*library.db"?

mrjohnpoz commented 3 years ago

maybe that is not actually the db?? searching for anywhere else the db might be with

but sofar only thing showing is this ash-4.4# find / -name com.plexapp.plugins.library.db /volume1/@appstore/PlexMediaServer/Resources/com.plexapp.plugins.library.db /volume1/docker/plexdata/com.plexapp.plugins.library.db

mrjohnpoz commented 3 years ago

Ok looks like its here /volume1/@apphome/PlexMediaServer/Plex Media Server/Plug-in Support/Databases/com.plexapp.plugins.library.db

Found this thread as well about the move https://www.reddit.com/r/synology/comments/o9wk33/dsm_7_and_plex_package_where_is_the_plex_database/

So guess have to change my symlink to there.. Sure hope there nothing changed in the db that tool could not like?

mrjohnpoz commented 3 years ago

hmmm - that is not a directoy??

ash-4.4# ls -la
total 25452
drwxr-xr-x 1 PlexMediaServer PlexMediaServer      390 Jul  9 16:36  .
drwxr-xr-x 1 root            root                 266 Jul 13 08:07  ..
-rwxr-xr-x 1 PlexMediaServer PlexMediaServer   683544 Jul  9 16:08  CrashUploader
drwxr-xr-x 1 PlexMediaServer PlexMediaServer     1808 Jul  9 16:14  lib
-rwxr-xr-x 1 PlexMediaServer PlexMediaServer   267240 Jul  9 15:45 'Plex Commercial Skipper'
-rwxr-xr-x 1 PlexMediaServer PlexMediaServer  2596280 Jul  9 16:05 'Plex DLNA Server'
-rwxr-xr-x 1 PlexMediaServer PlexMediaServer   113584 Jul  9 15:45 'Plex Media Fingerprinter'
-rwxr-xr-x 1 PlexMediaServer PlexMediaServer  3822056 Jul  9 16:09 'Plex Media Scanner'
-rwxr-xr-x 1 PlexMediaServer PlexMediaServer 15036512 Jul  9 16:14 'Plex Media Server'
-rwxr-xr-x 1 PlexMediaServer PlexMediaServer   651048 Jul  9 15:45 'Plex Relay'
-rwxr-xr-x 1 PlexMediaServer PlexMediaServer    10992 Jul  9 15:45 'Plex Script Host'
-rwxr-xr-x 1 PlexMediaServer PlexMediaServer    10680 Jul  9 15:45 'Plex SQLite'
-rwxr-xr-x 1 PlexMediaServer PlexMediaServer   302768 Jul  9 15:45 'Plex Transcoder'
-rwxr-xr-x 1 PlexMediaServer PlexMediaServer  2551656 Jul  9 15:45 'Plex Tuner Service'
drwxr-xr-x 1 PlexMediaServer PlexMediaServer     1962 Jul 16 19:22  Resources
drwxr-xr-x 1 PlexMediaServer PlexMediaServer       24 Jul  9 16:36  ui
ash-4.4#
mynttt commented 3 years ago

If you symlink /volume1/@apphome/PlexMediaServer/Plex Media Server/ to /plexdata you won't need the override.

I guess nothing has changed in the DB. But even if the tool would just crash because it couldn't handle the schema before doing anything.

mrjohnpoz commented 3 years ago

Ok - WTF did they do... I found this

ash-4.4# pwd
/var/packages/PlexMediaServer/home/Plex Media Server/Plug-in Support/Databases
ash-4.4# ls -la
total 4571292
drwx------+ 1 PlexMediaServer PlexMediaServer      1262 Jul 16 19:13 .
drwx------+ 1 PlexMediaServer PlexMediaServer       100 Mar 14  2018 ..
-rwx------+ 1 PlexMediaServer PlexMediaServer     69632 Sep 21  2019 com.plexapp.dlna.db
-rwx------+ 1 PlexMediaServer PlexMediaServer     32768 Sep 21  2019 com.plexapp.dlna.db-shm
-rwx------+ 1 PlexMediaServer PlexMediaServer    292552 Sep 21  2019 com.plexapp.dlna.db-wal
-rwx------+ 1 PlexMediaServer PlexMediaServer 667003904 Jul 16 19:10 com.plexapp.plugins.library.blobs.db
-rwx------+ 1 PlexMediaServer PlexMediaServer 659277824 Jul  5 00:01 com.plexapp.plugins.library.blobs.db-2021-07-05
-rwx------+ 1 PlexMediaServer PlexMediaServer 659555328 Jul  8 00:04 com.plexapp.plugins.library.blobs.db-2021-07-08
-rwx------+ 1 PlexMediaServer PlexMediaServer 662366208 Jul 11 00:05 com.plexapp.plugins.library.blobs.db-2021-07-11
-rwx------+ 1 PlexMediaServer PlexMediaServer 676015104 Jul 14 00:05 com.plexapp.plugins.library.blobs.db-2021-07-14
-rwx------+ 1 PlexMediaServer PlexMediaServer     32768 Jul 16 19:13 com.plexapp.plugins.library.blobs.db-shm
-rwx------+ 1 PlexMediaServer PlexMediaServer         0 Jul 16 19:13 com.plexapp.plugins.library.blobs.db-wal
-rwx------+ 1 PlexMediaServer PlexMediaServer 272334848 Jul 16 19:27 com.plexapp.plugins.library.db
-rwx------+ 1 PlexMediaServer PlexMediaServer 266928128 Jul  5 00:00 com.plexapp.plugins.library.db-2021-07-05
-rwx------+ 1 PlexMediaServer PlexMediaServer 268713984 Jul  8 00:03 com.plexapp.plugins.library.db-2021-07-08
-rwx------+ 1 PlexMediaServer PlexMediaServer 272248832 Jul 11 00:03 com.plexapp.plugins.library.db-2021-07-11
-rwx------+ 1 PlexMediaServer PlexMediaServer 271933440 Jul 14 00:04 com.plexapp.plugins.library.db-2021-07-14
-rwx------+ 1 PlexMediaServer PlexMediaServer     32768 Jul 16 20:00 com.plexapp.plugins.library.db-shm
-rwx------+ 1 PlexMediaServer PlexMediaServer   4152992 Jul 16 20:00 com.plexapp.plugins.library.db-wal
ash-4.4#
mynttt commented 3 years ago

Oh that's normal. It's backups and temporary files from sqlite3. And plex has multiple databases that cover different domains. But only one is interesting for this tool.

mrjohnpoz commented 3 years ago

So with their changes in permissions - should the docker now be set to run as this internal " PlexMediaServer" account?

mynttt commented 3 years ago

I think you can leave it as it is and try it out once you have the correct path set. I'm very sure that it will work out of the box now.

mrjohnpoz commented 3 years ago

ok looking good

NFO ] - 2021-07-17 01:23:12 @ ImdbPipeline.analyseDatabase: Resolving IMDB identifiers for items. Only warnings and errors will show up...
[INFO ] - 2021-07-17 01:23:12 @ ImdbPipeline.analyseDatabase: Items that show up here will not be processed by further stages of the pipeline.
[INFO ] - 2021-07-17 01:23:12 @ ImdbPipeline.analyseDatabase: Filtered 1 invalid item(s).
[INFO ] - 2021-07-17 01:23:13 @ ImdbPipeline.transformMetadata: 3037 item(s) need no update.
[INFO ] - 2021-07-17 01:23:13 @ ImdbPipeline.transformMetadata: Transforming 37 item(s)
[INFO ] - 2021-07-17 01:23:13 @ ImdbTransformer.updateMetadata: Adjust rating: 5.8 -> 5.7 for A Very Brady Sequel
[INFO ] - 2021-07-17 01:23:13 @ ImdbTransformer.updateMetadata: Adjust rating: 6.9 -> 6.8 for Pardon My Sarong
[INFO ] - 2021-07-17 01:23:13 @ ImdbTransformer.updateMetadata: Adjust rating: 6.8 -> 6.9 for Bud Abbott Lou Costello Meet the Killer Boris Karloff
[INFO ] - 2021-07-17 01:23:13 @ ImdbTransformer.updateMetadata: Adjust rating: 5.4 -> 5.5 for The Oak Room
[INFO ] - 2021-07-17 01:23:13 @ ImdbTransformer.updateMetadata: Adjust rating: 6.2 -> 6.3 for Here Today
[INFO ] - 2021-07-17 01:23:13 @ ImdbTransformer.updateMetadata: Adjust rating: 3.7 -> 3.6 for Mortal Kombat: The Journey Begins
mrjohnpoz commented 3 years ago

but still had to create a sym.. since docker gui will not let you pick anything that is not in share folder as mount point..

mrjohnpoz commented 3 years ago

So I created a symlink mount --bind "/var/packages/PlexMediaServer/home/Plex Media Server/Plug-in Support/Databases" /volume1/docker/plexdata

Mounted docker/plexdata (docker is a share I created).. with plexdata being a folder under docker.

Then using the override set db to /plexdata

Looks to be working - THANK YOU!!

That link will not survive a reboot of the nas.. But that is easiest enough to fix.. And also that almost never happens so could just create by hand again if needed..

mrjohnpoz commented 3 years ago

Ok something went wonky... Not exactly sure what - but after update ran, I couldn't play anything on plex.. My guess had to do with the symlink.. I rebooted the nas, since I knew that would remove the link.. And seems everything is working again.. Left the updatetool docker not running.. Leaving for vacation tmrw, and want my plex working - so going to leave it as is for now.. Not exactly sure what was going on.. But wouldn't play from one of my sticks nor could I get web to play.. Thought I was going to have to restore the db backup I just did.. But after reboot and symlink gone its working again.. hmmmm

mynttt commented 3 years ago

Docker and symlink seems to be wonky. Maybe you could try creating the symlink with the ln command so.

ln -s "/var/packages/PlexMediaServer/home/Plex Media Server/Plug-in Support/Databases" /volume1/docker/plexdata/db

And then set the override to /plexdata/db.

But other than that I have no idea what would cause this behavior.

mrjohnpoz commented 3 years ago

Yeah be sure to test it when I get back from my trip. Clearly the db was fine after updatetool ran, since its working now. And the changes it made are there.

I might be able to just mount that var/packages location with docker cmd line - its just the gui editing of the docker in synology does not allow you to mount that location. I will see what happens when I create the volume mount to var/packages path via cmd line in docker..

Plex was running - and could see everything, just when went to go play anything gave a error.

mrjohnpoz commented 3 years ago

Ok - got it working.. the gui in synology wasn't working. even trying ln, etc..

So did it via cmd line with, I just got rid of the volume mounts in the docker gui in synology.. And did them from cmd line 

ash-4.4# docker update -v "/var/packages/PlexMediaServer/home/Plex Media Server/Plug-in Support/Databases":/plexdata -v /volume1/docker/imdpupdaterconfig:/config confident_perlman

So doing an inspect of my docker I see them like this.

        "Mounts": [
            {
                "Type": "bind",
                "Source": "/volume1/docker/imdpupdaterconfig",
                "Destination": "/config",
                "Mode": "rw",
                "RW": true,
                "Propagation": "rprivate"
            },
            {
                "Type": "bind",
                "Source": "/var/packages/PlexMediaServer/home/Plex Media Server/Plug-in Support/Databases",
                "Destination": "/plexdata",
                "Mode": "rw",
                "RW": true,
                "Propagation": "rprivate"
            }
        ],

This seems to be working.. Will keep an eye on it, and see what happens when it runs again in 12 hours. But I can see it updated a bunch of stuff.. And plex is working - I started a few videos and they were working. See what happens when users try, etc.. 

INFO ] - 2021-07-23 13:00:17 @ ImdbTransformer.updateMetadata: Adjust rating: 7.2 -> 7.1 for The Better Bargain
[INFO ] - 2021-07-23 13:00:17 @ ImdbTransformer.updateMetadata: Adjust rating: 7.0 -> 6.9 for The Big Switch
[INFO ] - 2021-07-23 13:00:17 @ ImdbTransformer.updateMetadata: Adjust rating: 8.1 -> 8.2 for Ain't No Reason to Get Excited
[INFO ] - 2021-07-23 13:00:17 @ ImdbTransformer.updateMetadata: Adjust rating: 7.4 -> 7.6 for Pups Save a Herd
[INFO ] - 2021-07-23 13:00:17 @ ImdbPipeline.transformMetadata: Transformed entries for 1035 items(s).

Set the override to just /plexdata

mrjohnpoz commented 3 years ago

Just to follow up.. Tool seems to be working as it should now.. I for sure had plenty of people streaming yesterday with no complaints. And I watched quite a bit myself without issues.

And tool ran on schedule and updated more stuff. [INFO ] - 2021-07-24 13:58:57 @ ImdbPipeline.transformMetadata: Transformed entries for 11 items(s). [INFO ] - 2021-07-24 13:58:59 @ ImdbPipeline.updateDatabase: Updating 92 via batch request...

Back in business.. Thank you for your time and such a great tool..

mrjohnpoz commented 3 years ago

Just to close this up - plex has moved back to putting the db in the share path.. Version 1.24.2.4973

So back to the typical way of just mounting the share to /plexdata

But thanks again for this fix/workaround - kept tool working for quite sometime..