search

l3uddz / plex_autoscan

Script to assist sonarr/radarr with plex imports. Will only scan the folder that has been imported, instead of the whole library section.
GNU General Public License v3.0
396 stars 70 forks source link

Could not locate record in media_parts #35

Closed camjac251 closed 5 years ago

camjac251 commented 6 years ago

I've been having this error a lot with trying to use this script with Sonarr/Radarr. I'm not sure what is not working. I get the following error below and have tried to adjust my config to fix it but have had no luck

My config is

{
  "DOCKER_NAME": "user", 
  "PLEX_ANALYZE_DIRECTORY": true, 
  "PLEX_ANALYZE_TYPE": "basic", 
  "PLEX_DATABASE_PATH": "/folder/to/user/Library/Application Support/Plex Media Server/Plug-in Support/Databases/com.plexapp.plugins.library.db", 
  "PLEX_EMPTY_TRASH": true, 
  "PLEX_EMPTY_TRASH_CONTROL_FILES": [
    "/folder/user/gmedia/mountcheck"
  ], 
  "PLEX_EMPTY_TRASH_MAX_FILES": 100, 
  "PLEX_EMPTY_TRASH_ZERO_DELETED": false, 
  "PLEX_LD_LIBRARY_PATH": "/folder/to/user/Library/Plex/current/usr/lib/plexmediaserver", 
  "PLEX_LOCAL_URL": "http://localhost:38192", 
  "PLEX_SCANNER": "/folder/user/user/Library/Plex/current/usr/lib/plexmediaserver/Plex\\ Media\\ Scanner", 
  "PLEX_SECTION_PATH_MAPPINGS": {
    "1": [
      "/Movies/"
    ], 
    "2": [
      "/TV/"
    ]
  }, 
  "PLEX_SUPPORT_DIR": "/folder/to/user/Library/Application\\ Support", 
  "PLEX_TOKEN": "oo23m3om3o3mrotfj", 
  "PLEX_USER": "user", 
  "PLEX_WAIT_FOR_EXTERNAL_SCANNERS": true, 
  "RCLONE_RC_CACHE_EXPIRE": {
    "ENABLED": true, 
    "MOUNT_FOLDER": "/folder/user/gmedia", 
    "RC_URL": "http://user:k29r9v9k92k3e9fir9jgj9rj93@localhost:12341"
  }, 
  "RUN_COMMAND_BEFORE_SCAN": "", 
  "SERVER_ALLOW_MANUAL_SCAN": false, 
  "SERVER_FILE_EXIST_PATH_MAPPINGS": {}, 
  "SERVER_IGNORE_LIST": [
    "/.grab/", 
    ".DS_Store", 
    "Thumbs.db"
  ], 
  "SERVER_IP": "127.0.0.1", 
  "SERVER_MAX_FILE_CHECKS": 10, 
  "SERVER_PASS": "zyUX3ygQHAanrZQChPQAJQz5wLT850UwkE5BbnhWXIt52Agjc87Mr3RNvjIKGtNO", 
  "SERVER_PATH_MAPPINGS": {}, 
  "SERVER_PORT": 12345, 
  "SERVER_SCAN_DELAY": 5, 
  "SERVER_SCAN_PRIORITIES": {
    "0": [
      "/TV/"
    ], 
    "1": [
      "/Movies/"
    ]
  }, 
  "SERVER_USE_SQLITE": true, 
  "USE_DOCKER": false, 
  "USE_SUDO": false
}

And the server log

2018-06-04 16:29:14,591 -     INFO -  AUTOSCAN [id2]: Starting queue processor in 10 seconds
 2018-06-04 16:29:14,591 -     INFO -  AUTOSCAN [id]: Starting server:
 2018-06-04 16:29:24,599 -     INFO -  AUTOSCAN [id2]: Queue processor started
 2018-06-04 16:29:24,600 -     INFO -  AUTOSCAN [id2]: Restored 0 scan requests from database
"/home/user/folder/Library/Application Support/Plex Media Server/Plug-in Support/Databases/com.plexapp.plugins.library.db" 2018-06-04 16:30:34,725 -    DEBUG -  AUTOSCAN [id]: Client '127.0.0.1' request dump:
{
    "episodeFile": {
        "id": 11111, 
        "path": "/home/ms/files/deluge/Show.2015.S02.1080p.NF.WEB-DL.DD5.1.x264-PAs/Show.2015.S02E03.1080p.NF.WEB-DL.DD5.1.x264-PAs.mkv", 
        "quality": "WEBDL-1080p", 
        "qualityVersion": 1, 
        "relativePath": "Season 2/Show (2015) - S02E03 - WEBDL-1080p PAs.mkv", 
        "releaseGroup": "PAs", 
        "sceneName": "Show.2015.S02E03.1080p.NF.WEB-DL.DD5.1.x264-PAs"
    }, 
    "episodes": [
        {
            "airDate": "2017-01-01", 
            "airDateUtc": "2017-01-01T00:00:00Z", 
            "episodeNumber": 3, 
            "id": 11111, 
            "quality": "WEBDL-1080p", 
            "qualityVersion": 1, 
            "releaseGroup": "PAs", 
            "sceneName": "Show.2015.S02E03.1080p.NF.WEB-DL.DD5.1.x264-PAs", 
            "seasonNumber": 2, 
            "title": "The Title Name"
        }
    ], 
    "eventType": "Download", 
    "isUpgrade": false, 
    "series": {
        "id": 111, 
        "path": "/folder/user/gmedia/TV/Show (2015)", 
        "title": "Show (2015)", 
        "tvdbId": 111111
    }
}
 2018-06-04 16:30:34,725 -     INFO -  AUTOSCAN [id]: Client '127.0.0.1' scan request for series: '/folder/user/gmedia/TV/Show (2015)/Season 2/Show (2015) - S02E03 - WEBDL-1080p PAs.mkv', event: 'Download'
 2018-06-04 16:30:34,725 -    DEBUG -  AUTOSCAN [id]: Using section id: 2 for '/folder/user/gmedia/TV/Show (2015)/Season 2/Show (2015) - S02E03 - WEBDL-1080p PAs.mkv'
 2018-06-04 16:30:34,761 -     INFO -  AUTOSCAN [id]: Added '/folder/user/gmedia/TV/Show (2015)/Season 2/Show (2015) - S02E03 - WEBDL-1080p PAs.mkv' to database, proceeding with scan
 2018-06-04 16:30:34,763 -     INFO -      PLEX [id2]: Scan request from Sonarr for '/folder/user/gmedia/TV/Show (2015)/Season 2/Show (2015) - S02E03 - WEBDL-1080p PAs.mkv', sleeping for 5 seconds...
 2018-06-04 16:30:39,766 -     INFO -      PLEX [id2]: File '/folder/user/gmedia/TV/Show (2015)/Season 2/Show (2015) - S02E03 - WEBDL-1080p PAs.mkv' exists on check 1 of 10.
 2018-06-04 16:30:39,766 -    DEBUG -     UTILS [id2]: Using priority 0 for path '/folder/user/gmedia/TV/Show (2015)/Season 2'
 2018-06-04 16:30:39,767 -    DEBUG -      PLEX [id2]: Waiting for turn in the scan request backlog with priority: 0
 2018-06-04 16:30:39,767 -     INFO -      PLEX [id2]: Scan request is now being processed
 2018-06-04 16:30:39,787 -     INFO -      PLEX [id2]: No 'Plex Media Scanner' processes were found.
 2018-06-04 16:30:39,787 -     INFO -      PLEX [id2]: Starting Plex Scanner
 2018-06-04 16:30:39,787 -    DEBUG -      PLEX [id2]: export LD_LIBRARY_PATH=/home/user/folder/Library/Plex/current/usr/lib/plexmediaserver;export PLEX_MEDIA_SERVER_APPLICATION_SUPPORT_DIR=/home/user/folder/Library/Application\ Support;/home/user/folder/Library/Plex/current/usr/lib/plexmediaserver/Plex\ Media\ Scanner --scan --refresh --section 2 --directory '/folder/user/gmedia/TV/Show (2015)/Season 2'
 2018-06-04 16:30:40,335 -     INFO -     UTILS [id2]: GUI: Scanning Show Season 2 Episode 1
 2018-06-04 16:30:40,337 -     INFO -     UTILS [id2]: GUI: Scanning Show Season 2 Episode 2
 2018-06-04 16:30:40,342 -     INFO -     UTILS [id2]: GUI: Scanning Show Season 2 Episode 3
 2018-06-04 16:30:40,645 -     INFO -      PLEX [id2]: Finished scan!
 2018-06-04 16:30:40,667 -     INFO -      PLEX [id2]: Removed '/folder/user/gmedia/TV/Show (2015)/Season 2/Show (2015) - S02E03 - WEBDL-1080p PAs.mkv' from database
 2018-06-04 16:30:41,668 -     INFO -      PLEX [id2]: There is 0 queued items remaining...
 2018-06-04 16:30:41,668 -     INFO -      PLEX [id2]: Checking deleted item count in 10 seconds...
 2018-06-04 16:30:51,684 -     INFO -      PLEX [id2]: Skipping emptying trash as there were no deleted items
 2018-06-04 16:30:51,684 -    DEBUG -      PLEX [id2]: Sleeping 10 seconds before sending analyze request
 2018-06-04 16:31:01,698 -    ERROR -      PLEX [id2]: Could not locate record in media_parts where file = '/folder/user/gmedia/TV/Show (2015)/Season 2/Show (2015) - S02E03 - WEBDL-1080p PAs.mkv', 1/5 attempts...
 2018-06-04 16:31:11,701 -    ERROR -      PLEX [id2]: Could not locate record in media_parts where file = '/folder/user/gmedia/TV/Show (2015)/Season 2/Show (2015) - S02E03 - WEBDL-1080p PAs.mkv', 2/5 attempts...
 2018-06-04 16:31:21,706 -    ERROR -      PLEX [id2]: Could not locate record in media_parts where file = '/folder/user/gmedia/TV/Show (2015)/Season 2/Show (2015) - S02E03 - WEBDL-1080p PAs.mkv', 3/5 attempts...
 2018-06-04 16:31:31,717 -    ERROR -      PLEX [id2]: Could not locate record in media_parts where file = '/folder/user/gmedia/TV/Show (2015)/Season 2/Show (2015) - S02E03 - WEBDL-1080p PAs.mkv', 4/5 attempts...
 2018-06-04 16:31:41,727 -    ERROR -      PLEX [id2]: Could not locate record in media_parts where file = '/folder/user/gmedia/TV/Show (2015)/Season 2/Show (2015) - S02E03 - WEBDL-1080p PAs.mkv', 5/5 attempts...
 2018-06-04 16:31:51,730 -    ERROR -      PLEX [id2]: Could not locate record in media_parts where file = '/folder/user/gmedia/TV/Show (2015)/Season 2/Show (2015) - S02E03 - WEBDL-1080p PAs.mkv' after 5 tries...
 2018-06-04 16:31:51,731 -     INFO -      PLEX [id2]: Aborting analyze of '/folder/user/gmedia/TV/Show (2015)/Season 2/Show (2015) - S02E03 - WEBDL-1080p PAs.mkv' because could not find any metadata_item_id for it
l3uddz commented 6 years ago

This means the path for the file it just scanned could not be found in Plex. I would check the path of another episode from the same season in Plex and compare it with the path the script is looking for. If it's different, you need to change the Plex server path mapping config option so it turns the path sonarr reports to the path Plex will see and understand.

camjac251 commented 6 years ago

For some reason the paths aren't different. I did this. Let sonarr process an episode and copy to the show directory, sonarr sends that to autoscan, and autoscan shows

Finished Scan
Removed 'path'
Could not locate record

I then did scanned within Plex itself and it found the episode, so I copied that path and the path given by sonarr, they end up being the same. Same as what plex_autoscan outputs as well. I'm not sure what is going on.

In my config, you see the \ for spaces for the Plex paths, but is it needed if the paths could be put in double quotes? I'm on Gentoo (linux), is it needed for python with Windows?

l3uddz commented 6 years ago

Config paths look fine, the episodes wouldn't appear in Plex once autoscan had scanned them if they were wrong, I'd imagine. Are you sure the Plex database path is correct?

camjac251 commented 6 years ago

Yup, it is. I tried backslashes to the db path but it gave me the error that the DB couldn't be found so I took them out and it didn't give the error anymore.

l3uddz commented 6 years ago

Very strange... Maybe look in the database itself and see if you can find the media file record it's looking for. I'm assuming the files are scanning into Plex fine, its just unable to find it in the Plex db to initiate the analysis, so that's where I would look next.

camjac251 commented 6 years ago

The analysis happens after Plex is able to see the file and has added it to the library right? For me, plex doesn't display any of the episodes it scans in from plex autoscan on the library. Should it show in the library at this point? Could the "SERVER_USE_SQLITE": true, be the problem if it is unable to add the entries to the database?

l3uddz commented 6 years ago

That setting shouldn't have ant effect on Plex scans.

You can see in the log it executed the scanner and Plex scanned 3 episodes, at this point, the new episode should appear in plex, and autoscan will then be able to find the metadata item id for that new episode inside the database to launch the analysis mode from the Plex CLI scanner.

camjac251 commented 6 years ago

I tried running the command that plex_autoscan gives back manually to see if I got a different result

export LD_LIBRARY_PATH="/folder/to/Library/Plex/current/usr/lib/plexmediaserver";export PLEX_MEDIA_SERVER_APPLICATION_SUPPORT_DIR="/folder/to/user/Library/Application Support";"/folder/to/user/Library/Plex/current/usr/lib/plexmediaserver/Plex Media Scanner" --scan --refresh --section 2 --directory "/folder/to/gmedia/TV/Show's Title Name/Season 2"

is what I ended up running. Although it gave back GUI: Scanning Show's Title Name Season 2 Episode 1 through 7 but the one that Sonarr sorted, 7 doesn't appear.

camjac251 commented 6 years ago

I think I was able to get it fixed. Whatbox uses this as their startup script for Plex

#!/bin/sh
#SCRIPTPATH=$(dirname $(python -c 'import sys,os;print os.path.realpath(sys.argv[1])' $0))
SCRIPT=$(readlink -f $0)
SCRIPTPATH=`dirname ${SCRIPT}`
export LD_PRELOAD="/usr/local/lib/erfolg.so"
export LD_LIBRARY_PATH="${SCRIPTPATH}"
export PLEX_MEDIA_SERVER_HOME="${SCRIPTPATH}"
export PLEX_MEDIA_SERVER_MAX_PLUGIN_PROCS=6
export LC_ALL="en_US.UTF-8"
export LANG="en_US.UTF-8"
ulimit -s 3000
./Plex\ Media\ Server

I think PLEX_MEDIA_SERVER_APPLICATION_SUPPORT_DIR might need to be removed from my config and PLEX_MEDIA_SERVER_HOME (with LD_PRELOAD possibly too) added in its place pointing to the same path as LD_LIBRARY_PATH in order for it to work strangely. Would the "RUN_COMMAND_BEFORE_SCAN" option work for this kind of thing? So like "RUN_COMMAND_BEFORE_SCAN"=export LD_PRELOAD="/usr/local/lib/erfolg.so";export PLEX_MEDIA_SERVER_HOME="/folder/to/user/Library/Plex/current/usr/lib/plexmediaserver And commenting out PLEX_SUPPORT_DIR in the config to remove the PLEX_MEDIA_SERVER_APPLICATION_SUPPORT_DIR environment?

l3uddz commented 6 years ago

I'd try updating the LD library path to the same as tht script sets first. and pointing the application support dir to the correct location. That should hopefully do it.

l3uddz commented 6 years ago

You could try the export in run command before config option but not positive it will work, iirc someone else had whatbox issues and it was resolved iirc by setting the correct locations fr the existing variables.

camjac251 commented 6 years ago

I had a question on the new command that's ran every 60 seconds. I still haven't been able to get it working yet but I feel like I'm close.

'Plex Media Scanner' is running, pid: , cmdline: ['/folder/to/Library/Plex/current/usr/lib/plexmediaserver/Plex Media Scanner', '--scan', '--refresh', '--section', '2']. Checking again in 60 seconds... With this running, does it run with the ld library env path before? On my regular shell, I get the error error while loading shared libraries: libboost_atomic.so.1.59.0: cannot open shared object file: No such file or directory when just running "/folder/to/Library/Plex/current/usr/lib/plexmediaserver/Plex Media Scanner" --scan --refresh --section 2 but if I run it with defining the ld library ahead, it works, like this

LD_LIBRARY_PATH="/folder/to/Library/Plex/current/usr/lib/plexmediaserver" "/folder/to/Library/Plex/current/usr/lib/plexmediaserver/Plex Media Scanner" --scan --refresh --section 2.

Does plex_autoscan do this by default. I can't seem to tell when running it even in the debug log level. It just says that its running it every 60 seconds but doesn't output the log from the process it runs.

camjac251 commented 6 years ago

I found the cause of the problem but it isn't something I think I can fix myself.

In order for the plex scanner to work on this server I'm on, export LD_PRELOAD="/usr/local/lib/erfolg.so" needs to be added to export LD_LIBRARY_PATH=/folder/to/Library/Plex/current/usr/lib/plexmediaserver;export PLEX_MEDIA_SERVER_APPLICATION_SUPPORT_DIR=/folder/to/Library/Application\ Support;/folder/to/Library/Plex/current/usr/lib/plexmediaserver/Plex\ Media\ Scanner --scan --refresh --section 2 --directory '/folder/to/rclone/TV/Show/Season 1' Like this

export LD_PRELOAD="/usr/local/lib/erfolg.so";export LD_LIBRARY_PATH=/folder/to/Library/Plex/current/usr/lib/plexmediaserver;export PLEX_MEDIA_SERVER_APPLICATION_SUPPORT_DIR=/folder/to/Library/Application\ Support;/folder/to/Library/Plex/current/usr/lib/plexmediaserver/Plex\ Media\ Scanner --scan --refresh --section 2 --directory '/folder/to/rclone/TV/Show/Season 1'

For some reason RUN_COMMAND_BEFORE_SCAN="export LD_PRELOAD=/usr/local/lib/erfolg.so" doesn't work in this case with it, still gives the error from before.

camjac251 commented 6 years ago

My solution right now is adding it to the end of the PLEX_LD_LIBRARY_PATH like this "PLEX_LD_LIBRARY_PATH": "/folder/to/Library/Plex/current/usr/lib/plexmediaserver;export LD_PRELOAD=/usr/local/lib/erfolg.so;export PLEX_PORT=12345", and so far it works.

TehRobot commented 4 years ago

I have the same issue but I think mine is due to AutoScan running on windows and the media being on a UnRaid server.

Aborting analysis of '\10.1.2.115\Media\TV Shows\9-1-1 (2018)\Season 2/9-1-1.S02E10.Merry.Ex-Mas.HDTV-1080p.MeGusta.mkv' because could not find any 'metadata_item_id' for it.

Where it adds the incorrect / to the file path.

edrock200 commented 4 years ago

What is the erfolg.so file you are referring to preloading? It's not in my Plex and I'm having a similar issue with srt's being picked up.

camjac251 commented 4 years ago

That was some specific to my distro of linux. I no longer use it though.

dh0llyw00d commented 4 years ago

What is the erfolg.so file you are referring to preloading? It's not in my Plex and I'm having a similar issue with srt's being picked up.

You can find the environment variables that the plex scanner is using by following these steps:

My solution right now is adding the missing environment variables to the PLEX_LD_LIBRARY_PATH like this "PLEX_LD_LIBRARY_PATH": "/home/username/Library/Plex/current/usr/lib/plexmediaserver;export LD_PRELOAD=/usr/local/lib64/erfolg.so;export PLEX_PORT=12345"

Thanks to @camjac251 for the path to success

WoisWoi commented 3 years ago

Hi,

I've the same problem, but your solution doesn't work for me : image