Plex DB Size growing exponentially

[Animosity022]

I had recently noticed that once I started and converted my other things to plexapi calls, my db growth has been growing at quite an alarming rate. I was up to 2.5GB of DB size and eventually things crawled to halt.

I have this post on the plex forums that I just posted:

https://forums.plex.tv/t/database-size-growth-when-using-plexapi/585315

That has more of the size information and what I did to fix the issue in the short term.

My use case is pretty simple as I have two scripts:

• one runs every 5 minutes and empties the trash with some checking with number of items as I use cloud storage and tie this into my automated setup
• second runs every 15 minutes during the day from from 8am-11pm to mark shows watched as it has a list of shows to mark

I am just trying to figure out if I'm doing some abnormal or it's a plexapi issue or a plex db issue.

[Hellowlol]

I dont think this is a plexapi issue. All we do is a http call to plex that handles the rest You can manually compare the the url we call vs what plex webs does and see if they do anything different. With that being said your use case seems kinda strange.

Deleting items don’t seems to do really do that much, unless you remove old metadata (cleanbundels) and use optimize db from time to time. What scanner settings have you enabled?

[Animosity022]

It's pretty easy for me to reproduce as I can narrowed it to the script I'm running to mark shows watched.

felix@gemini:/var/lib/plexmediaserver/Library/Application Support/Plex Media Server/Plug-in Support/Databases$ sqldiff --summary com.plexapp.plugins.library.db.fixed com.plexapp.plugins.library.db | grep meta | grep view
metadata_item_views: 0 changes, 1965 inserts, 0 deletes, 531 unchanged

Each time I run that, I get 2k inserts in that view table which balloons up as I running it every 5 minutes.

My script isn't anything magical as I just wanted to mark a bunch of stuff watched routinely so it runs through a bunch of these:

plex.library.section('TV Shows').get('The Bachelor').markWatched()

as an example.

It very well could be I'm using it wrong as well as perhaps running these too often has side effects.

[jjlawren]

Do you repeatedly re-mark the same shows as watched throughout the day? Is it necessary to empty the trash every 5 minutes? These both seem very excessive.

I'm not sure I understand the use case where it's desired to programmatically change the watched status instead of just using the actual watch history from normal client use. Are you using "watched" as some other kind of indicator?

[Animosity022]

Yes, I repeatedly mark the same thing, which might be the problem as I was using more of a 'hammer' approach to just ensuring they were marked.

For the second the trash to be more specific, I check to see if anything is in the trash and empty it. So that might actually only fire a handful of times in a day.

As for the use-case, take the show the Bachelor. I have a family friend that watches it and we do not watch it in my house. I never want to it to show up as Unwatched as that gets messed in with my other shows. I want a certain set of shows to automatically be watched. If you try to do it in manually as new ones show up, they appear on deck. You can do it via the regular methods but I was looking for something more automatic.

If my hammer approach is truly the root cause and was hoping to get some validation on that, I can remove the hammer and do a cleaner approach and only mark when something new appears and mark that item as watched.

Sorry if any of this isn't clear as my brain to written English on Monday seems bad today.

[jjlawren]

As @hellowlol said, I can't see how this could be caused by anything "bad" that plexapi is doing, but just misuse of standard Plex functionality. You'll never see other users with this problem as the native Plex clients won't/can't behave this way.

Maybe you could rearrange things and create a "Bad TV Shows" library that you could hide for yourself. 😄

[Animosity022]

I think you would see the problem but it's unlikely as a user it's manually going to click 25 shows and marked them watched every 15 minutes for 14 or so hours a day would be well outside the norm since the API is just mimicking what happens from the client.

I am on the same page as in chatting it out here and on the plex forum sides, it's a user created error by me :)

Appreciate the help and suggestions and prompt replies as I'll close it out as not a plexapi issue either.

[Hellowlol]

Why not use a separate account for your friend? If you really want to use a hammer you can check for any unwatched episodes in a ignore list and only mark the latest episode as watched. I dont think this will trigger that big inserts

[Animosity022]

They do use separate accounts already as that's why I want to mark it watched in my account so I never see it.

I was going to do just that as I realize the hammer was a bit much as I was being lazy.

I was going to follow your approach and build something a bit more clean like:

• keep a list of shows that I can add to
• check and build of a list of any unwatched episodes
• mark only those episodes as unwatched

as my feeling is the bulk marks is what kills it.

Since I know what to look for, it's very easy to test now.

[Hellowlol]

I have never noticed what someone else have watched on deck. Are you using managed account?

[Animosity022]

I use my unplayed state in my TV shows to stroll through new things.

The bad show is continuing so once I mark the first few seasons watched, they eventually show back up when a new season/episode air.

I used to use https://github.com/itscontained/AutoMarkWatched but it doesn't work anymore so I moved to PlexAPI to handle it and just botched my initial setup.

So it's all accounts and such are setup properly and things work great, I just like to mark stuff I don't watch and will never watch as played so it doesn't show up ever on my on deck or my unplayed shows.

[jjlawren]

Why not put the shows you'll never care about in a second TV Shows library?

[Animosity022]

I'd have to move them in Sonarr once they were auto added via Ombi, adjust them in Plex and a few more steps to make my other automation didn't break.

Updating one line in a mark watched "seemed" easier and it was even easier with AutoMarkWatched.

Plus, I occasionally get the wife factor and she watches something I've deemed dumb so she'd look in the normal spot not find it and I would get questions on where is it.

I try to do things on the backside so keep things simple for the family/friends as I'm a nice guy :)

[blacktwin]

You could use labels but labels are whitelist not blacklist. So you'd need at least 2 labels, "For me" and "Not for me". Everything could get both expect anything you don't want would only get "Not for me" and your user would need the "For me" label. Your wife wouldn't be able to see it on your account though. Also this is assuming you have Plex Pass.

[Animosity022]

That's a really good idea as well. I'll tell a look at that @blacktwin and yes I do have Plex Pass.

Let me look into that a bit more and see how I can make that work.

I truly appreciate all the great ideas coming in as well!

[Animosity022]

What I finally got and I need to finish up the code for it was a much cleaner approach using a shared label.

I basically am going to to the following:

• tv search using my label for AutoMark
• check those shows for unwatched item
• if they are unwatched, get the unwatched episodes
• mark each episode watched

That way I am only ever marked an episode would should be pretty clean.

I got my label working as I just need to code up the rest of it as it's all pretty new to me.

Thanks for the great idea though as it's so much cleaner and easier to do and you can see that stuff right in the WebApp with a filter on.

[Animosity022]

Just in case anyone does have a use case like mine, I ended up with small script.

#!/usr/bin/python3

import os
import subprocess
import sys
os.environ['PLEXAPI_CONFIG_PATH'] = "/home/felix/scripts/plexapi.ini"

from plexapi.server import PlexServer
plex = PlexServer()

tv = plex.library.section('TV Shows')
for shows in tv.search(label='AutoWatch'):
    for episode in shows.unwatched():
        episode.markWatched()

That way it only marks things when it has to.