l3uddz / plex_autoscan

Script to assist sonarr/radarr with plex imports. Will only scan the folder that has been imported, instead of the whole library section.
GNU General Public License v3.0
395 stars 70 forks source link

Support updating files #67

Closed saitoh183 closed 5 years ago

saitoh183 commented 5 years ago

When i update a show or movie with a better version, since the name is the same, plexscan doesnt seem to tell plex to update. Like i had a movie that had only english audio, i replaced it with a dual audio version. The file name stayed the same but i was forced to go in plex and launch a manual analyze to get the file info to change

desimaniac commented 5 years ago

This is strictly Google Drive Monitoring or is this with webhooks?

saitoh183 commented 5 years ago

Gdrive monitoring

desimaniac commented 5 years ago

Ok. Currently it checks to see if the file is already included in Plex, and if so, it ignores the scan request altogether. We'll have to look into this..

saitoh183 commented 5 years ago

Thanks

Animosity022 commented 5 years ago

I use this and replace files all the time. Does the log not show an analyze starting? Mine does when I check on a replaced file.

saitoh183 commented 5 years ago

i will check next time i have a file to replace

saitoh183 commented 5 years ago

so i gave it a try again and i tried to update a movie that was only english to dual audio (french and english) and the logs says it saw it but nothing happened on plex side.

Currrent plex: image image

all i see in log:

2019-01-11 00:28:43,451 -     INFO -  AUTOSCAN [140412929455872]: There's 1 Google Drive change(s) to process
 2019-01-11 00:28:43,497 -     INFO -  AUTOSCAN [140412929455872]: Ignored 1 file(s) from Google Drive changes for already being in Plex!
 2019-01-11 00:38:49,046 -     INFO -  AUTOSCAN [140412929455872]: There's 1 Google Drive change(s) to process
 2019-01-11 00:38:49,050 -     INFO -  AUTOSCAN [140412929455872]: Ignored 1 file(s) from Google Drive changes for disallowed file extensions
 2019-01-11 00:39:49,665 -     INFO -  AUTOSCAN [140412929455872]: There's 1 Google Drive change(s) to process
 2019-01-11 00:39:49,667 -     INFO -  AUTOSCAN [140412929455872]: Ignored 1 file(s) from Google Drive changes for disallowed file extensions

the new file is in the folder...notice the size and modified date:

image

I also tried to delete file duplicate and renaming a file in hopes it would update plex accordingly but it didnt for the duplicate deletion.

  2019-01-11 00:15:35,955 -     INFO -  AUTOSCAN [140412929455872]: There's 2 Google Drive change(s) to process
 2019-01-11 00:15:35,956 -     INFO -  AUTOSCAN [140412929455872]: Removed '1mHPGmRk4L4VMY1VAiFxUUf0ghaYMUxQH' from cache: The House with a Clock in Its Walls (2018) WEBDL-1080p.mp4
 2019-01-11 00:15:35,957 -     INFO -  AUTOSCAN [140412929455872]: Removed '1OKN0XlU5zjWXgxtO1QnVgKtvmRbnPvfK' from cache: The Old Man & the Gun (2018) WEBDL-720p.mp4
 2019-01-11 00:17:37,098 -     INFO -  AUTOSCAN [140412929455872]: There's 1 Google Drive change(s) to process
 2019-01-11 00:17:37,151 -     INFO -  AUTOSCAN [140412929455872]: Ignored 1 file(s) from Google Drive changes for already being in Plex!
 2019-01-11 00:21:39,295 -     INFO -  AUTOSCAN [140412929455872]: There's 1 Google Drive change(s) to process
 2019-01-11 00:21:39,348 -     INFO -  AUTOSCAN [140412929455872]: Proceeding with scan of 1 file(s) from Google Drive changes: [u'My Drive/Videos/Movies/The Old Man & the Gun (2018)/The Old Man & the Gun (2018) Bluray-1080p.mp4']
 2019-01-11 00:21:39,355 -     INFO -  AUTOSCAN [140412929455872]: Added '/media/gdrive_encrypted/Videos/Movies/The Old Man & the Gun (2018)/The Old Man & the Gun (2018) Bluray-1080p.mp4' to database, proceeding with scan
 2019-01-11 00:21:39,355 -     INFO -      PLEX [140412905379584]: Scan request from Google Drive for '/media/gdrive_encrypted/Videos/Movies/The Old Man & the Gun (2018)/The Old Man & the Gun (2018) Bluray-1080p.mp4', sleeping for 180 seconds...
 2019-01-11 00:24:39,469 -     INFO -      PLEX [140412905379584]: File '/media/gdrive_encrypted/Videos/Movies/The Old Man & the Gun (2018)/The Old Man & the Gun (2018) Bluray-1080p.mp4' exists on check 1 of 10.
 2019-01-11 00:24:39,470 -     INFO -      PLEX [140412905379584]: Scan request is now being processed
 2019-01-11 00:24:39,526 -     INFO -      PLEX [140412905379584]: No 'Plex Media Scanner' processes were found.
 2019-01-11 00:24:39,526 -     INFO -      PLEX [140412905379584]: Starting Plex Scanner
 2019-01-11 00:24:41,020 -     INFO -  AUTOSCAN [140412929455872]: There's 1 Google Drive change(s) to process
 2019-01-11 00:24:41,070 -     INFO -  AUTOSCAN [140412929455872]: Ignored 1 file(s) from Google Drive changes for already being in Plex!
 2019-01-11 00:24:42,430 -     INFO -     UTILS [140412905379584]: GUI: Scanning The Old Man & the Gun (2018)
 2019-01-11 00:24:43,577 -     INFO -     UTILS [140412905379584]: GUI: Background media analysis on The Old Man & the Gun
 2019-01-11 00:24:44,805 -     INFO -      PLEX [140412905379584]: Finished scan!
 2019-01-11 00:24:44,807 -     INFO -      PLEX [140412905379584]: Removed '/media/gdrive_encrypted/Videos/Movies/The Old Man & the Gun (2018)/The Old Man & the Gun (2018) Bluray-1080p.mp4' from database
 2019-01-11 00:24:45,809 -     INFO -      PLEX [140412905379584]: There is 0 queued items remaining...
 2019-01-11 00:24:55,821 -     INFO -      PLEX [140412905379584]: Found row in media_parts where file = '/media/gdrive_encrypted/Videos/Movies/The Old Man & the Gun (2018)/The Old Man & the Gun (2018) Bluray-1080p.mp4' after 1/5 tries!
 2019-01-11 00:24:55,822 -     INFO -      PLEX [140412905379584]: Starting basic analysis of metadata_item(s): 81228
 2019-01-11 00:24:56,251 -     INFO -     UTILS [140412905379584]: GUI: Media analysis on The Old Man & the Gun
 2019-01-11 00:24:57,351 -     INFO -      PLEX [140412905379584]: Finished basic analysis of metadata_item(s): 81228!

the file removed was The House with a Clock in Its Walls (2018) WEBDL-1080p.mp4

but in plex..: image

desimaniac commented 5 years ago

This behavior won't be changed after all. A lot of users of Plex Autoscan use it with local scanning. But they also use Google Drive Monitoring to catch anything that was added to Gdrive directly and therefore not scanned locally. GDM sends a lot of requests for "newly found files" this way and, therefore, most of these files are already in Plex. So this is why it is checked against Plex to see if it already exists, and if so, is ignoring. And this is something we can't just check file size and modification time for.

What I can suggest for you is maybe change the naming style you use; if you are modifying the audio, maybe add those tags to the filename. The name change would definitely trigger a new scan of the item.

Edit: clarification

saitoh183 commented 5 years ago

Sorry but i dont understand your explanation. But for the second file why didnt it delete it like Plex does when you use monitoring feature in plex. the files arent the same name. As for the files that get replaced but have the same name. this should atleast trigger an analysis which would update the file information and not rescan it into the library

Animosity022 commented 5 years ago

I'm still not following what the problem you are having is.

I replace files all the time and use the same file names. Plex analyzes them via the CLI.

I just replaced a 720 download with a 1080p here:

1-14 08:47:14,102 -     INFO -      PLEX [140279636858624]: Trash cleared for section 2 after 1/5 tries!
 2019-01-14 08:47:24,112 -     INFO -      PLEX [140279636858624]: Found row in media_parts where file = '/gmedia/TV/Victoria/Victoria.S03E01.mkv' after 1/5 tries!
 2019-01-14 08:47:24,112 -     INFO -      PLEX [140279636858624]: Starting basic analysis of metadata_item(s): 27560
 2019-01-14 08:47:24,283 -     INFO -     UTILS [140279636858624]: GUI: Media analysis on Victoria S03 E01
 2019-01-14 08:47:24,330 -     INFO -      PLEX [140279636858624]: Finished basic analysis of metadata_item(s): 27560!
saitoh183 commented 5 years ago

Notice above in my previous post, Old man & the Gun file is 4.3GB in plex. I replaced it with a dual audio version 8.3GB file and plex didnt update it

Animosity022 commented 5 years ago

But Plex did update it:

 2019-01-11 00:24:55,821 -     INFO -      PLEX [140412905379584]: Found row in media_parts where file = '/media/gdrive_encrypted/Videos/Movies/The Old Man & the Gun (2018)/The Old Man & the Gun (2018) Bluray-1080p.mp4' after 1/5 tries!
 2019-01-11 00:24:55,822 -     INFO -      PLEX [140412905379584]: Starting basic analysis of metadata_item(s): 81228
 2019-01-11 00:24:56,251 -     INFO -     UTILS [140412905379584]: GUI: Media analysis on The Old Man & the Gun
 2019-01-11 00:24:57,351 -     INFO -      PLEX [140412905379584]: Finished basic analysis of metadata_item(s): 81228!

And analyzed it.

I'm assuming you want Plex to empty the trash:

  "PLEX_EMPTY_TRASH": true,

So the older movie is removed.

saitoh183 commented 5 years ago

ah ok, i will try that...maybe that is what is missing

l3uddz commented 5 years ago

Assuming this is resolved. The file not showing as available in Plex would have indeed be resolved by PLEX_EMOTY_TRASH being set to true (as it would have removed the file it just upgraded)

saitoh183 commented 5 years ago

sorry i have been away and did have access to my Pc. But no the problem is still there even with the Plex_Empty_Trash. I still have to do an analyse manually if i replace a file that has already been scanned in. Plex_autoscan doesnt seem to do anything. I will post a log in a few minutes

Animosity022 commented 5 years ago

I'm not sure what you mean.

Do you me scan the file and add it to the library? Do you mean run a analyze against the file? Do you mean the old file isn't deleted?

Do you have a log showing what you are trying to explain?

saitoh183 commented 5 years ago

@animosity22

im hoping this will clear up the problem:

The example i will use is Mortal Engines (2018)

So Radarr downloaded a all english version and Plex_autoscan added it...so far all is good. Now 11 days later, i wanted to change the movie for a dual audio version (french/english).

So the first screenshots is the current state of things: image image

The second screenshot is Radarr showing i downloaded a new version: image

The third screenshots is after i download and replaced the file: image image

Notice in Plex the metadata did not update. The file is now 2GB and has dual audio.

The log, nowhere does it say it updated the movie. plex_autoscan.log

And obviously when you replace a file it will keep the same name. The only line in the log that shows it saw the file (i think..) is 2019-02-24 18:37:11,749 - INFO - AUTOSCAN [140335353911040]: Ignored 1 file(s) from Google Drive changes for already being in Plex!

I hope this explanation is clearer :)

Animosity022 commented 5 years ago

Yes, your settings are still messed up along the way as your logs don't show that movie being added at all by plex autoscan.

Here is an example of 2 upgrades of Creed 2 from my logs and you can see it kept the same file name all 3 times.

 2019-02-12 00:50:56,294 -     INFO -  AUTOSCAN [140032988804864]: Client '127.0.0.1' scan request for movie: '/gmedia/Radarr_Movies/Creed II (2018)/Creed II (2018).mkv', event: 'Download'
 2019-02-12 00:50:56,295 -     INFO -      PLEX [140032922527488]: Scan request from Radarr for '/gmedia/Radarr_Movies/Creed II (2018)/Creed II (2018).mkv', sleeping for 5 seconds...
 2019-02-12 00:51:01,299 -     INFO -      PLEX [140032922527488]: File '/gmedia/Radarr_Movies/Creed II (2018)/Creed II (2018).mkv' exists on check 1 of 10.
 2019-02-12 00:51:01,610 -     INFO -     UTILS [140032922527488]: GUI: Scanning Creed Ii (2018)
 2019-02-12 00:51:01,617 -     INFO -     UTILS [140032922527488]: GUI: Matching 'Creed Ii'
 2019-02-12 00:51:09,911 -     INFO -     UTILS [140032922527488]: GUI: Score for 'Creed II' (2018) is 105
 2019-02-12 00:51:09,926 -     INFO -     UTILS [140032922527488]: GUI: Requesting metadata for 'Creed II'
 2019-02-12 00:51:09,942 -     INFO -     UTILS [140032922527488]: GUI: Background media analysis on Creed II
 2019-02-12 00:51:30,657 -     INFO -      PLEX [140032922527488]: Found row in media_parts where file = '/gmedia/Radarr_Movies/Creed II (2018)/Creed II (2018).mkv' after 1/5 tries!
 2019-02-12 00:51:30,852 -     INFO -     UTILS [140032922527488]: GUI: Media analysis on Creed II
 2019-02-19 14:22:52,645 -     INFO -  AUTOSCAN [140354543945152]: Client '127.0.0.1' scan request for movie: '/gmedia/Radarr_Movies/Creed II (2018)/Creed II (2018).mkv', event: 'Upgrade'
 2019-02-19 14:22:52,645 -     INFO -      PLEX [140354477651712]: Scan request from Radarr for '/gmedia/Radarr_Movies/Creed II (2018)/Creed II (2018).mkv', sleeping for 5 seconds...
 2019-02-19 14:22:57,652 -     INFO -      PLEX [140354477651712]: File '/gmedia/Radarr_Movies/Creed II (2018)/Creed II (2018).mkv' exists on check 1 of 10.
 2019-02-19 14:22:57,906 -     INFO -     UTILS [140354477651712]: GUI: Scanning Creed Ii (2018)
 2019-02-19 14:22:57,924 -     INFO -     UTILS [140354477651712]: GUI: Background media analysis on Creed II
 2019-02-19 14:23:18,012 -     INFO -      PLEX [140354477651712]: Found row in media_parts where file = '/gmedia/Radarr_Movies/Creed II (2018)/Creed II (2018).mkv' after 1/5 tries!
 2019-02-19 14:23:18,087 -     INFO -     UTILS [140354477651712]: GUI: Media analysis on Creed II
 2019-02-23 00:44:02,967 -     INFO -  AUTOSCAN [140354543945152]: Client '127.0.0.1' scan request for movie: '/gmedia/Radarr_Movies/Creed II (2018)/Creed II (2018).mkv', event: 'Upgrade'
 2019-02-23 00:44:02,967 -     INFO -      PLEX [140353508185856]: Scan request from Radarr for '/gmedia/Radarr_Movies/Creed II (2018)/Creed II (2018).mkv', sleeping for 5 seconds...
 2019-02-23 00:44:07,973 -     INFO -      PLEX [140353508185856]: File '/gmedia/Radarr_Movies/Creed II (2018)/Creed II (2018).mkv' exists on check 1 of 10.
 2019-02-23 00:44:08,290 -     INFO -     UTILS [140353508185856]: GUI: Scanning Creed Ii (2018)
 2019-02-23 00:44:08,309 -     INFO -     UTILS [140353508185856]: GUI: Background media analysis on Creed II
 2019-02-23 00:44:28,391 -     INFO -      PLEX [140353508185856]: Found row in media_parts where file = '/gmedia/Radarr_Movies/Creed II (2018)/Creed II (2018).mkv' after 1/5 tries!
 2019-02-23 00:44:28,508 -     INFO -     UTILS [140353508185856]: GUI: Media analysis on Creed II

The log you included shows that Mortal Engines was never added by plex autoscan and never asked to scan it.

desimaniac commented 5 years ago

Let me try to explain it for him šŸ˜› (assuming he is a he šŸ˜¬ )

File is scanned into Plex during first import, but... he modifies the file's audio but keep the original name. Google Drive Monitor sends a request. Plex Autoscan check to see if the file is there already, and says " this file already exists in Plex, Plex Autoscan will now ignore it".

This protection is there because if we didn't have it, Plex would be continuously rescanning everything Google Drive would throw at PAS...

So.. my suggestion would be to maybe add something to the filename that would show you updated the file, maybe a new audio tag, or whatever. But I checked with l3uddz on this, and there is pretty much nothing that can be done here.

Animosity022 commented 5 years ago

Yeah, he's not using Radarr to send the request to Plex as that is how the workflow show be if Radarr is upgrading it.

saitoh183 commented 5 years ago

here is my config config.txt

I just noticed a line i forgot to change. could it be the reason?

"PLEX_LD_LIBRARY_PATH": "/usr/lib/plexmediaserver", since my plex library is moved , shouldnt it be change?

[Service] Environment="PLEX_MEDIA_SERVER_APPLICATION_SUPPORT_DIR=/data/Library/Application Support"

Also i dont use webhook, only, only Google drive monitoring

saitoh183 commented 5 years ago

Also adding something to the file name isnt a good idea because then i would end up with something like this in plex

image

Animosity022 commented 5 years ago

Yeah, your whole setup doesn't make sense to me.

You want to use the tool effectively and use the webhooks so you can take advantage of it and use the empty trash.

If you keep the file names the same, how it is going to know something changed? Keep a database of each file and run info against it?

l3uddz commented 5 years ago

If you use the plex empty trash config option, it would remove the old file that is no longer there (your screenshot)

saitoh183 commented 5 years ago

@animosity22

how does my setup not make sense? i just use gdrive monitoring which i assume doesnt need the webhook part because it works just fine for adding content. So how are you suppose to use Gdrive monitoring only when you update files. It is not like you can rename the file differently because radarr/sonarr are doing the renaming base on your naming scheme.

@l3uddz

for the screenshot above of instant family, this is normal. Since i use gdrive with rclone encrypt, I could not point plex_autoscan to my gdrive that plex uses. So since i have a backup gdrive, not encrypted, I point plex_autoscan to that instead. It works because i always transfer the file to the non-encrypted gdrive last which gives enough time for the file to appear in the encrypted mount so that when plex_autoscan launches its detection, the file is present for plex to see. Yes it is not conventional but it was the only way i was able to get plex_autoscan to work when having an encrypted mount.

For Instant family i deleted the file on the non-encrypted drive first (my mistake) which would trigger autoscan but for plex, the file is still there because i didnt delete it on the encrypted mount. .

saitoh183 commented 5 years ago

If there is no solution to this problem, i will just accept the fact that i need to launch Analyse in Plex manually every time i update a file. It is a pain to do but if there is no other option, then so be it. :(

Animosity022 commented 5 years ago

He gave you a solution.

Change the file name. Enable empty trash for Plex.

Animosity022 commented 5 years ago

Or use the webhooks and keep the file name the same.

saitoh183 commented 5 years ago

@animosity22

Changing the file name isn't a solution. Changing filename would mean manual intervention which is the same as launching the analyse in plex. When using Gdrive monitoring, how is this handled? You have to use webhook as well to cover when you update a file? I know better version would have the quality changed so it is a different file but when audio is changed I would be forced to add audio to naming which I would have like to avoid.

Animosity022 commented 5 years ago

No, I just use a webhook with Radarr and Sonarr and never touch a thing.

I also use plex empty trash for when I delete shows as well.

desimaniac commented 5 years ago

Emptying of plex trash had nothing to do with how you mounted it. It would have just removed anything that was ā€œunavailableā€, as in your screenshot.

saitoh183 commented 5 years ago

@desimaniac

Are you talking about my explanation into why I have a file unavailable?

desimaniac commented 5 years ago

When you rename a file, and plex scans it in via PAS, the old filename will be listed there as ā€œunavailableā€. PAS will then empty the trash and remove the ā€œunavailableā€ items from Plex. And the old filename media will not be listed there anymore.

I think I get why you are not seeing this behavior. Empty out the control file section.

"PLEX_EMPTY_TRASH_CONTROL_FILES": [
],
saitoh183 commented 5 years ago

When you rename a file, and plex scans it in via PAS, the old filename will be listed there as ā€œunavailableā€. PAS will then empty the trash and remove the ā€œunavailableā€ items from Plex. And the old filename media will not be listed there anymore.

I think I get why you are not seeing this behavior. Empty out the control file section.

"PLEX_EMPTY_TRASH_CONTROL_FILES": [
],

i never set that...i left what was there by default.

"PLEX_EMPTY_TRASH": true, 
  "PLEX_EMPTY_TRASH_CONTROL_FILES": [
    "/mnt/unionfs/mounted.bin"
  ], 
saitoh183 commented 5 years ago

I just noticed a line i forgot to change. could it be the reason?

"PLEX_LD_LIBRARY_PATH": "/usr/lib/plexmediaserver", since my plex library is moved , shouldnt it be change?

[Service] Environment="PLEX_MEDIA_SERVER_APPLICATION_SUPPORT_DIR=/data/Library/Application Support"

What about this?

desimaniac commented 5 years ago

@saitoh183 https://github.com/l3uddz/plex_autoscan/commit/844eb3931d20dffedc03411311dbdc2ac70cbcb9