Not deleting unplayed media past creation data criteria

[Doug411]

Hi again! I have a folder that I am populating with podcasts. I'm using TubeArchivist to download them to a Podcast Library. Each individual podcast is a series folder on Jellyfin. Many of the podcasts are daily. I'd like to set a retention period for 14 days, and delete old content regardless of whether it has been watched or not (unless its favorited), I've tried various incarnations, but the only items it seems to put in my delete queue are items that were watched by the monitored user.

This is what I thought would work based on my understanding of the logic. (turn off played evaluation, set my created condition, and put the count <0). However, it only flagged the played items.

episode: played: condition_days: -1 count_equality: '>=' count: 1 created: condition_days: 14 count_equality: '>=' count: -1 behavioral_control: true

Also, what is the best way to potentially change the retention period at the podcast level? eg

Podcast (Jellyfin Series Library) -Podcast 1 (Sub Folder) 14 days -Podcast 2 (Sub Folder) 21 days -Podcast 3 (Sub Folder) 30 days

I also want to clear up my understanding of how the script works. If I use tags, do I still have to blacklist the library that I'm evaluating? In other words is it possible to NOT blacklist the podcast library, but blacktag items in a library that is whitelisted?

What happens if I tag the individual podcast folders, versus tagging the items? Tubeachivist doesn't have functionality to tag items upon download. I'm trying to figure out the best way to accomplish a folder based retain/purge logic. ideas?

I also want to understand more why you are allowing multiple monitor users. If I monitor 5 users played status.... Is the main use case to not delete until all 5 have watched and it meets the days criteria? Wow, that's complex if so.

[terrelsa13]

I think I figured it out. Jellyfin and Emby return tag data differently. Somehow for Jellyfin if a child item did not already have a tag it was skipping over inheriting the tag(s) from the parent.

Try v.5.8.15-beta

[Doug411]

https://pastebin.com/BtcLNGKD

sorry about the last paste. I'll try v5.8.15.

[Doug411]

We're making progress. it runs to completion for all users. Runs on both series libraries. However, it didn't flag any items for deletion. Like something is wrong with played logic? I moved the creation to 4 days, and couldn't get it to flag anything for deletion.

[terrelsa13]

Apologies. This was a me induced problem.

Same compare function as before. My previous fixes swung the pendulum too far in the other direction. This time it was taking the individual letters from a string and comparing them to the partner item.

Instead of comparing "abc123" to "abc123".

It was comparing... "a" to "abc123" "b" to "abc123" . . . "3" to "abc123"

I have double confirmed the compare function works for all current use cases.

advanced_settings > fitler_statements Update

A handful of betas ago, the advanced_settings > filter_statement layout was updated. MUMC will still understand the way it was. But the new layout decouples blacklisting/whitelisting from favorites/tags; allowing the possibility to query emby/jellfin for less data. Go ahead and delete the old section and replace with the updated section. This will keep you current with the most up-to-date config layout:

DELETE THIS SECTION

advanced_settings:
  filter_statements:
    episode:
      query_filter:
        #query whitelists for media_items meeting the basic_settings > filter_statement > episode > created query
        #query whitelists for blacktagged media_items
        favorited: false
        whitetagged: false
        blacktagged: true
        whitelisted: true
        blacklisted: false

REPLACE WITH THIS UPDATED SECTION

advanced_settings:
  filter_statements:
    episode:
      query_filter:
        whitelisted:
          #Do not look for favorited media_items in whitelisted libraries
          favorited: false
          #Do not look for whitetagged media_items in whitelisted libraries
          whitetagged: false
          #Look for blacktagged media_items in whitelisted libraries
          blacktagged: true
          #Do not look for watched media_items in whitelisted libraries
          whitelisted: false
        blacklisted:
          #Look for favorited media_items in blacklisted libraries
          favorited: true
          #Do not look for whitetagged media_items in blacklisted libraries
          whitetagged: false
          #Look for blacktagged media_items in blacklisted libraries
          blacktagged: true
          #Look for watch media_items in blacklisted libraries
          blacklisted: true

Then give v5.8.16-beta a spin!

[Doug411]

Outstanding! I 'm feeling lucky.....:) I'll let you know in a few minutes.

Interesting regarding your substring compare. We still don't have a way to substring compare on the jellyfin API side, do we? Because that would be and interesting solution for our future tagging enhancement.. ( MUMC_created14>=0)... If we could request all items with tags that started with MUMC , then we wouldnt need the tags in the yaml, would we?, It would just parse the tag when it returns the full string in python? But i'm guessing the jellyfin API doesnt support any substring or regex queries, which is what I think you indicated previously.

[Doug411]

Functionally it seems to be working.(although there may be a problem)..... But....wow! what a difference on performance! I remonitored both of my show libraries, and the changes you made to filtering made a huge impact. It only queries jellyfin for a small fraction of what it was before. What was taking several minutes to finish is now well under a minute.

A couple of quick questions:

1. On the summary, its showing some deletions from the regular shows library (Bill Mahr). However, it is not showing the items from the YouTube library that are flagged for deletion in the detail above.

2. FYI...I had a bunch of content created on may 31. Its saying that was 8 days ago. I'm not bothered by it, but just wondering about the calculation.

3. What was the flag to keep x minimum number of episodes? Can I use that in conjuction with my existing logic (eg keep 7 episodes). I would assume it would delete > 14 days created, if i have it configured the way I do now, unless that gets me below 7 episodes? Is that the way it would work?

4. Before I turn it on for real (to actually do the deletions), I just want to understand what happens at the end. So it goes through all my people/episodes and lets me know by person which episodes will be kept and which will be deleted.. Is the final step to combine all those sections together? EG, if anyone is keeping, it keeps the episode (and ignores the record from other people that may suggest it should be deleted)?

5. I sent you an email with the detail from my screen so you can compare the summary at the end to the detail

[terrelsa13]

Interesting regarding your substring compare. We still don't have a way to substring compare on the jellyfin API side, do we? Because that would be and interesting solution for our future tagging enhancement.. ( MUMC_created14>=0)... If we could request all items with tags that started with MUMC , then we wouldnt need the tags in the yaml, would we?, It would just parse the tag when it returns the full string in python? But i'm guessing the jellyfin API doesnt support any substring or regex queries, which is what I think you indicated previously.

Correct there is no way I am aware of to search for partial tags or perform some kind of regex query. But I will also say, I am by no means an expert in the Emby/Jellfyin APIs. So it could be me not Googling right thing to find out how to do it.

Functionally it seems to be working.(although there may be a problem)..... But....wow! what a difference on performance! I remonitored both of my show libraries, and the changes you made to filtering made a huge impact. It only queries jellyfin for a small fraction of what it was before. What was taking several minutes to finish is now well under a minute.

Good good. Also setting DEBUG: 0 speeds things up a lot because the script does not have to stop and write information to the debug log.

There are also console controls config variables that allow hiding some of the things being printed to the screen. This does give a performance boost because the script does not have to stop and print things on the screen. BUT, there's no point in messing with that until we have things up and running correctly.

On the summary, its showing some deletions from the regular shows library (Bill Mahr). However, it is not showing the items from the YouTube library that are flagged for deletion in the detail above.

I think you may be talking about this: Per-User Delete/Keep vs True Delete/Keep

I sent you an email with the detail from my screen so you can compare the summary at the end to the detail

We did set a lot of the libraries to lib_enabled: false earlier. Make sure the libraries you want enabled are set back to lib_enabled: true. This might be why it seemed like it was running so much faster. It wasn't really looking at all of the libraries for all of the users.

FYI...I had a bunch of content created on may 31. Its saying that was 8 days ago. I'm not bothered by it, but just wondering about the calculation.

Emby/Jellyfin store all time using the UTC time zone. Could be this. Or it could be the media_item returning a creation date that does not match when it was actually created.

You can check the creation time using this API query in a browser:

http://localhost:8096/Users/74d15d7a864441ccbabfc7122b9136af/Items/5ee0c8b58777f5de4ba3af1cc392abb2?enableImages=False&enableUserData=True&Fields=ParentId,Genres,Tags,RecursiveItemCount,ChildCount,Type&api_key=YOUR_API_KEY_HERE

This should pull info for the following media_item:

:*[DELETE] - Episode - Real Time with Bill Maher - s22.e17 - May 31, 2024: John Waters, David Axelrod, Ken Buck - HBO - Unplayed - Play Count: 0 - Created 8 days ago - Favorite: False - Whitetag: False - Blacktag: True - Whitelisted: True - Blacklisted: False - EpisodeID: 5ee0c8b58777f5de4ba3af1cc392abb2

There should be a line that says DateCreated:. Does this date match what you expect? Don't forget, UTC timezone.

You can check a different media_item by changing 5ee0c8b58777f5de4ba3af1cc392abb2 to the EpisodeID of the media item you want to look at.

What was the flag to keep x minimum number of episodes? Can I use that in conjuction with my existing logic (eg keep 7 episodes). I would assume it would delete > 14 days created, if i have it configured the way I do now, unless that gets me below 7 episodes? Is that the way it would work?

I think you are asking about this: Minimum Episodes vs Minimum Played Episodes??

If yes, then yes you understand what it does.

Before I turn it on for real (to actually do the deletions), I just want to understand what happens at the end. So it goes through all my people/episodes and lets me know by person which episodes will be kept and which will be deleted.. Is the final step to combine all those sections together? EG, if anyone is keeping, it keeps the episode (and ignores the record from other people that may suggest it should be deleted)?

1. filter_statements - get media_items from each user matching the defined pattern
2. behavioral_statements - compare media_items across all monitored_users and either add or remove media_items from the list of media_items to be deleted.
3. media_items are deleted
   1. When REMOVE_FILES: false; the script skips sending the delete request to emby/jellfyin for each media_item.
   2. When REMOVE_FILES: true, the script sends the delete request for each media_item to emby/jellyfin.
4. Emby/Jellyfin make the media_items disappear!

When REMOVE_FILES: true the summary looks like this:

----------------------------------------------------------------------------------------
Summary Of Deleted Media:
----------------------------------------------------------------------------------------
* Items Deleted = 111    *
----------------------------------------------------------------------------------------
[DELETED]   Episode - Aaahh!!! Real Monsters - s01.e06 - Monstrous Makeover - 13427128
[DELETED]   Episode - Aaahh!!! Real Monsters - s01.e07 - A Wing and a Scare - 13427137
.
.
.

When REMOVE_FILES: false the summary looks like this:

----------------------------------------------------------------------------------------
Summary Of Deleted Media:
----------------------------------------------------------------------------------------
* Dry Run Mode
* advanced_settings > REMOVE_FILES: false
* No Media Deleted
* Items = 111
----------------------------------------------------------------------------------------
To delete media, open mumc_config.yaml in a text editor:
----------------------------------------------------------------------------------------
* Set advanced_settings > REMOVE_FILES: true
----------------------------------------------------------------------------------------
[DELETED]   Episode - Aaahh!!! Real Monsters - s01.e06 - Monstrous Makeover - 13427128
[DELETED]   Episode - Aaahh!!! Real Monsters - s01.e07 - A Wing and a Scare - 13427137
.
.
.

[Doug411]

Thanks. I added the libraries/users back in the last run. The terminal printout is in the last attachment I sent to you from earlier today.

I have two series folders (Shows, YouTube) across 5 users. In the shows folder it flags some Bill Mahr episodes for deletion. In the Youtube folder it flags some episodes for deletion in the detailed printout (a few JRE and PBD episodes). However in the final summary the only episodes that it adds to the summary for deletion are the Bill Mahr episodes (nothing from the YouTube library). Seems odd, unless I'm not understanding something.

[Doug411]

FYI... Only 2 libraries have favorites and they are under my id (Doug) and they are just in PBD/JRE folders. I have one favorie in the JRE under the admin user. I just added a couple of favorites to test.

I have 6 total series under YouTube. The library is whitelisted and all 6 series are blacktagged at the series level. None of the episodes were in the deletion summary, but many were flagged for deletion as it was running through my library by person/episode and logging them as keep or delete in the terminal.

[Doug411]

I reran with a 5 day retention, and still nothing shows for deletion in the YouTube folder

[terrelsa13]

Let's simplify things to help us figure out what is going on.

1. Make a copy of the current config. Name the copy, mumc_config.yaml.orig
2. Open mumc_config.yaml in a text editor
3. Delete all users and their libraries that are not you (aka leave the Doug user info as-is)
4. Log into Jellyfin using the your account (Doug)
5. Unfavorite PBD and JRE + any seasons or episodes that may be favorited
6. Re-run the script

Is what you expect to be deleted showing as deleted at the end?

[Doug411]

Sent the log. This time it didn't add the Bill Mahr episodes from the shows library to the summary of deleted items (unless I missed it).

No items from the YouTube library were added again.

Again, its flagging items for deletion, but they aren't making it to the final summary

[Doug411]

Maybe I don't understand how that bottom summary should work, but it looks to me like it is saying there is nothing to delete.

[terrelsa13]

You are correct. It is not pretending to delete the files you are expecting. The summary should show the media_items that will be deleted at the end.

BUT I am seeing episodes favorited. So we need remove all other users from the config to try to understand if these favorited episodes are the issue and the ones you are expecting to be deleted. Or if there are others.

When I do a search for "IsFavorite": true, in the debug5.log, I see these PBD and Bill Maher episodes are favorited media_items.

Real Time with Bill Maher - S21E14 - Episode 634 HDTV-1080p.mkv
Real Time with Bill Maher - S22E14 - Episode 658 HDTV-1080p.mkv
Real Time with Bill Maher - S22E18 - Episode 662 HDTV-1080p.mkv
PBD Podcast | Ep 419

We want to completely remove all accounts except your Doug account from the config:

1. Rename the current mumc_config.yaml to mumc_config.yaml.orig
2. Run the script and create a new config; ONLY select your Doug account and whitelist libraries
   1. Do not select the admin account. Do not select any other users
3. Once the new config is created; open mumc_config.yaml.orig
4. Copy/Paste the basic_settings and advanced_settings from mumc_config.yaml.orig into mumc_config.yaml
5. In mumc_config.yaml set DEBUG: 4
6. Log into Emby/Jellyfin using your Doug account
7. Make sure no libraries, series, seasons, or episodes are favorited
8. Run the script

What do you see now?

Go ahead and send (attach or pastebin) both the config and debug log.

[Doug411]

ok. I'll need to rerun the setup script. I was running Jellyfin with Authentik/LDAP authentication. Just enabled OIDC and it created a new user account in jellyfin, tied to same record in Authentik. I ended up deleting my original ID (Doug) from Jellyfin after it was said and done. I assume deleting the account also deleted the associated metadata like favorites? Regardless I will just monitor the new account.

[Doug411]

Seems to work with the one account no favorites

[Doug411]

Which feels pretty good....first time I've seen it... since I started the journey... :) Victory is around the corner....

[terrelsa13]

I assume deleting the account also deleted the associated metadata like favorites?

Yes that is my understanding of how it works for Emby/Jellyfin.

Ok. Good to hear!

Now we take this one user at a time and see if/when we stop seeing what we are expecting to see.

1. In mumc_config.yaml change UPDATE_CONFIG: false to UPDATE_CONFIG: true
2. Run the script
3. Follow the steps and add ONE additional user; and whitelist their libraries
4. Finish the config updater
5. In mumc_config.yaml change UPDATE_CONFIG: true to UPDATE_CONFIG: false
6. There should now be TWO users in the config: Doug (or whatever name was used) and the new user just added (I am going to call the new user, USER1)
7. In the Emby/Jellyfin GUI; login as USER1
8. Make sure USER1 does not have any libraries, series, seasons, or episodes favorited
9. Run the script

What do you see now?

Go ahead and send (attach or pastebin) both the config and debug log.

[Doug411]

Ok. sorry been out of pocket for a few days....

I ran it a few times with a few different configs in order to isolate the problem.

The current logic works when all users have access to exactly the same content. When they don't, the logic will suggest for deletion only the items they have in common.

Problems:

1. If users have access to the same libraries, but at the item level, one user doesn't have access to that item (eg. via parental controls, there is a hard-error and it crashes)
2. If one user does not have access to a library and the other one does, then that library is excluded from the list of deletions. In my case both users had access to TV shows, and only 1 user had access to podcasts. therefore only tv shows made it to deletions. Only after I both granted access to podcasts to the second user, and whitelisted it for said user, did the program include those items in the list of deletions

[Doug411]

i was just doing some saturday home chores..... and remembered you had a concept of 'any' and 'all'. I have to go pick up my son... but got to thinking maybe we dont have that configured correctly. Maybe, I should have 'any user' somewhere instead of 'all user'... Regardless, there is definately an issue at the item level if some items are not accessible by a user (eg parental controls). I know that was broken a while ago, but i thought you fixed it. I'll look a little more when I get back

[Doug411]

Now that I think about it more, if tags arent user secific, it doesn't seem like any or all should matter. Is that just a favoriting concept?

[terrelsa13]

Now that I think about it more, if tags arent user secific, it doesn't seem like any or all should matter. Is that just a favoriting concept?

Correct. Tags are not user specific. any is not an option for them. If any is entered the script should generate an error that says something like, "all is a valid value".

but got to thinking maybe we dont have that configured correctly.

This very well could be it.

Help me out and correct me if I am wrong... The goal, if I remember correctly is to delete podcasts from the youtube library after they are at least 14 or more days old. Is this still true? Am I missing anything? Did anything change?

there is definately an issue at the item level if some items are not accessible by a user (eg parental controls). I know that was broken a while ago, but i thought you fixed it. I'll look a little more when I get back If users have access to the same libraries, but at the item level, one user doesn't have access to that item (eg. via parental controls, there is a hard-error and it crashes)

I did not do anything related to parental controls. The script has no knowledge of parental controls. The script is expecting each users configured for a library to have access to all media_items returned from queries to that library. I am willing to bet I have not heard of anyone running into the parental controls issues before because they either do no monitor the "child user". Or the "kid's shows" and "adult's shows" are in two different libraries; the "child user" does not have access to the "adult's shows" library.

I am looking at my library and I can see the parental ratings are not consistent within series, seasons, episodes. There are also some media_items in my library that have no rating. Filtering by rating (aka parental controls) does not seem like a consistent path.

I suggest removing parentally controlled users from the config.

If one user does not have access to a library and the other one does, then that library is excluded from the list of deletions. In my case both users had access to TV shows, and only 1 user had access to podcasts. therefore only tv shows made it to deletions. Only after I both granted access to podcasts to the second user, and whitelisted it for said user, did the program include those items in the list of deletions

Let's try this:

• Update to the latest v5.8.20-beta
• Set DEBUG: 1
• Run the script.
• Attach or Pastebin the debug log

This will allow me to see what you are seeing on the console. Then from the debug log, you can tell me which media_items you are expecting to be deleted but are not. Then I will have a better idea for additional questions I should be asking to get us to where we want to be

Question 1: Do all of the currently monitored_users NEED to be monitored to get the results you are looking for? Question 2: Are any of the media_items you are expecting to be deleted favorted or whitetagged?

[Doug411]

Nothing has changed in what I'm trying to accomplish. I'll rerun per your request. But the logic issue is what I described above. All my libraries are white listed. If I have one library (in this case podcasts), that some users dont have access to. They dont have access to it in jellyfin and they are not monitoring it in the script (they are monitoring other libraries though). The library (podcasts) will not be included in the deletions. Evn though I am monitoring it, have access to it, have no favorites, and it's items match the filter criteria and behavior criteria. If I grant all users access to that library in jellyfin, and have them monitor it in the script, then those entries get included in deletions. As I stated it seems that unless all users have access to the same library it gets excluded from deletions. That doesnt seem right.

[terrelsa13]

Try adding the dynamic_behavior variable to blacktagged episode under behavioral_statements.

advanced_settings:
  behavioral_statements:
    episode:
      blacktagged:
        dynamic_behavior: true

[Doug411]

FYI, just sent the logs and config email/drive (so I don't have to scrub). That was before I saw the comment you just wrote. I'll rerun with dynamic behavior: true

[Doug411]

I think that might have worked! Finally the podcast items are considered in the behavioral statement.

I read the description for that flag (I wouldn't have guessed from the description, that would be the fix). It led me to believe that it had something to do with played status (and I'm not considering played status, just created)

I'll play with it a little more tomorrow. I didn't give it good testing... But for the first time, it appears on the surface to be doing exactly what I want (deleting items created more than 14 days ago, regardless of played status. I'll have to add a few favorites to the mix and see what happens, etc)

Thanks for staying with it. Much appreciated.

[terrelsa13]

I think that might have worked! Finally the podcast items are considered in the behavioral statement. I'll play with it a little more tomorrow. I didn't give it good testing... But for the first time, it appears on the surface to be doing exactly what I want (deleting items created more than 14 days ago, regardless of played status. I'll have to add a few favorites to the mix and see what happens, etc)

Awesome!

I read the description for that flag (I wouldn't have guessed from the description, that would be the fix). It led me to believe that it had something to do with played status (and I'm not considering played status, just created)

There's a "played" for the played filter_statements and a "played" for the created filter_statements.

Looks like I need to update the wiki information for the dynamic_behavior variable.

Thanks for staying with it. Much appreciated.

Yeah yeah. Np! Let's not get too excited until you've had a chance to throw favorites into the mix to see how things behave.

[Doug411]

RE:

There's a "played" for the played filter_statements and a "played" for the created filter_statements....

Is the played under created truly tracking whether its played or not?

I added some favorites for monitored and unmonitored users. Expected results matched actual results. Core functionality seems to work well. Performance is really good.

I may have uncovered a new issue though. I added:

advanced_settings: episode_control: minimum_episodes: 10

which resulted in: Traceback (most recent call last): File "/opt/mumc/mumc_modules/mumc_minimum_episodes.py", line 254, in get_minEpisodesToKeep deleteItem['IndexNumber'] = int(deleteItem['IndexNumber']) ValueError: invalid literal for int() with base 10: '??'

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/opt/mumc/./mumc.py", line 156, in MUMC() File "/opt/mumc/./mumc.py", line 119, in MUMC deleteItems_dict=init_postProcessing(cfg) File "/opt/mumc/mumc_modules/mumc_post_process.py", line 570, in init_postProcessing deleteItems_dict=start_postProcessing(the_dict,episode_dict,deleteItems_dict) File "/opt/mumc/mumc_modules/mumc_post_process.py", line 517, in start_postProcessing deleteItems_media=postProcessing(the_dict,media_dict) File "/opt/mumc/mumc_modules/mumc_post_process.py", line 486, in postProcessing postproc_dict=get_minEpisodesToKeep(postproc_dict,the_dict) File "/opt/mumc/mumc_modules/mumc_minimum_episodes.py", line 262, in get_minEpisodesToKeep appendTo_DEBUG_log('\nItem[\'Id\'] : ' + str(deleteItem['Id']) + ' Skipped likley due to ParentIndexNumber ' + deleteItem['ParentIndexNumber'] + ' Not Being An Integer.',3,the_dict) TypeError: can only concatenate str (not "int") to str

[Doug411]

Also, getting close to running this for real. Where do you stand on the docker version? I'm considering running that. I'll probably lock to a specific version tag versus using latest. I update containers automatically on a schedule with watchtower. This seems like something that I would want to test first, versus upgrading to future releases automatically, given that its core functionality is deleting content. Happy to help test for you in the future, if you let me know.

[terrelsa13]

I may have uncovered a new issue though. I added:

advanced_settings: episode_control: minimum_episodes: 10

Try v5.8.21-beta with minimum_episodes: 10 again. Error should be fixed.

Also, getting close to running this for real. Where do you stand on the docker version? I'm considering running that. I'll probably lock to a specific version tag versus using latest. I update containers automatically on a schedule with watchtower. This seems like something that I would want to test first, versus upgrading to future releases automatically, given that its core functionality is deleting content. Happy to help test for you in the future, if you let me know.

For Docker you already did the hardest part, which is making a config. As long as the config is in one of these locations, should be no different than running native.

[terrelsa13]

Is the played under created truly tracking whether its played or not?

Yes, the created-played is checking if the media_item is played and how many times the media_item has been played.

[Doug411]

closer.... When I have episode control on, it no longer results in a hard error, however, it goes back to suggesting nothing to delete. I moved the integer down as low as 3, (and each one of my blacktagged series has at least 15 episodes. If I remove episode control, I get the correct deletions suggested, with it on, regardless of the number of episodes, I get 0 suggested deletions.

I just looked at your updated docker instructions. nice. Can I use a version tag like [v5.8.21-beta] with your docker solution, or am i constrained to just "latest" and "dev"? Once i get it going I'd like to lock onto a version.

Under docker scheduling, Interesting your run command...not sure I understand the python portion of the command.... I was planning on using compose to create the container, and then in cron, just "docker restart mumc"

[terrelsa13]

what value is being used for minimum_episodes_behavior?

  episode_control:
    minimum_episodes_behavior: ?

With multiple monitored_users the script needs to be configured to know how to determine the number of episodes to delete vs keep. If user1 has watched all episodes, user2 has watched half of the episodes, and user3 has watched none of the episodes; which user determines how many episodes should remain? minimum_episodes_behavior gives several configurable options.

[Doug411]

No, I didnt have that set. Interesting. I think I dont fully understand how its working after seeing that. I set it, and initially got another hard error. Then I realized I also needed to set minimum_played_episodes, which I set to 0. Then when setting episode behavior to any combination of Min/Max Played/Unplayed I would get no items suggested for deletion. However, if I put my user id into the minimum_episode_behavior, I get the same hard error I was getting before I set the Minimum_played_Episode value=0

PROCESSING MINIMUM NUMBER EPISODES...
Traceback (most recent call last):
  File "/opt/mumc/./mumc.py", line 156, in <module>
    MUMC()
  File "/opt/mumc/./mumc.py", line 119, in MUMC
    deleteItems_dict=init_postProcessing(cfg)
  File "/opt/mumc/mumc_modules/mumc_post_process.py", line 571, in init_postProcessing
    deleteItems_dict=start_postProcessing(the_dict,episode_dict,deleteItems_dict)
  File "/opt/mumc/mumc_modules/mumc_post_process.py", line 518, in start_postProcessing
    deleteItems_media=postProcessing(the_dict,media_dict)
  File "/opt/mumc/mumc_modules/mumc_post_process.py", line 487, in postProcessing
    postproc_dict=get_minEpisodesToKeep(postproc_dict,the_dict)
  File "/opt/mumc/mumc_modules/mumc_minimum_episodes.py", line 500, in get_minEpisodesToKeep
    if (unplayed_status['UserData']['Played']):
TypeError: 'bool' object is not subscriptable

Maybe I cant get what I want with this parameter. My basic understanding of your order of operations was that the program would go through each user and based on settings in config, determine which episodes it would keep or delete by user. Then I thought the program somehow merged that together; by looking at each episode (and in my case) if two users suggested keep it would keep, if both users suggest delete it would delete; but if one user suggested delete and one suggested keep (ie based on favorites), it would keep. then I assumed it would apply the minimum episode logic. Before your response, I thought it was just going to sum the number of deletions per series, subtract that from the total episodes for the series to get the proposed remaining episodes. EG (20 episodes total for a series, 12 proposed for deletion, would leave me with 8 episodes. If I set min_episodes to 10, I assumed it would throw out the last two deletion suggestions based on creation date (eg first in first out), which would bring me back to 10 episodes. Based on looking at the definitions of those parameters I suspect I was wrong in my assumptions. And I'm not sure I understand. It almost looks as if its going to always center on a single user to determine which episodes to delete. So if user A favorites an episode, but based on Minimum_episode_behavior user B is the one it selects..... And if User B doesn't have that episode favorites, the episode would get deleted. Which is not what I want. I wanted to retain 14 days worth of items regardless of played status. Keep all favorites for any user. and I thought it would be nice to set a minimum to not drop below 10 episodes for a series (ie podcasts that are 5 days a week, wouldn't fall below 2 weeks). Maybe I cant get what I want with this parameter, and I should let it go?

[Doug411]

FYI, I saw episode control as a nice to have. When I discovered it in the documentation, I thought it would be a nice add. If it could work the way I thought, I would use it (because not all my podcasts create content every day, etc) Keeping a constant 10 episodes seemed like a polished/consistancy touch. However if it compromises my ability to retain favorites for all users, then I would just not use it. I just want to understand if its worth pursuing. Right now when I turn it on, either it removes all deletions or hard errors depending on how set episode behavior. That might be a bug... But just based on the fact that it is even looking at a specific user in the alogrythm, I am drawing the conclusion that it may not be designed to do what I thought?

[terrelsa13]

Apologies. Lots of life things happening over my way.

Ok, episode control is generating an error. I tried this with Emby and was not able to get the error. I am going to try it with Jellyfin later tonight.

Then I thought the program somehow merged that together; by looking at each episode (and in my case) if two users suggested keep it would keep, if both users suggest delete it would delete; but if one user suggested delete and one suggested keep (ie based on favorites), it would keep. then I assumed it would apply the minimum episode logic.

That is partially how it works. behavior_statement logic (this includes favorites) is executed before episode_control logic. The *[DELETE] and [KEEPING] shown for individual users during the "get media" phase is just a guesstimate of what would happen to each media_item if they were only looked at in the bubble of the current user they are showing for. There is no way to know if a media_item will actually be deleted or kept until all media_items from all users are fetched. Once all media_items are fetched behavior_statements are run comparing individual media_items across each user. The script then knows which movies/episodes/etc... are going to be deleted. Then the episode_control logic can be run.

If the episode_control logic ran before behavior_statements the behavior_statement configuration would either add or remove episodes from the delete list causing the number of desired episodes remaining to be incorrect.

EG (20 episodes total for a series, 12 proposed for deletion, would leave me with 8 episodes. If I set min_episodes to 10, I assumed it would throw out the last two deletion suggestions based on creation date (eg first in first out), which would bring me back to 10 episodes.

Created date does not directly effect episode_controls; it would be indirectly related. The season and episode number determine which 2 episodes would remain in your example above. In this case the two episodes from the highest numbered season with the highest episode number are kept (i.e. the two newest episodes are kept).

Yes, if there are 20 total episodes and 12 are in the delete list, then at the end of episode_control, 2 or more episodes could be kept. I say, "2 or more" because the minimum_episodes_behavior considers the number of watched and unwatched episodes for each user. Depending on how minimum_episodes_behavior is configured will determine how many episodes remain.

If user1 watches episodes in a series faster than user2, and the script only considered user1; then when user2 went to watch episodes at the slower pace, episodes they have not watched yet will be deleted.

If user1 watches episodes in a series faster than user2, and the script only considered user2; it would appear this would be the right answer, because episodes would be deleted at the slower watchers pace. Except; is user2 slower at watching episodes? OR does user2 have no desire to watch any episodes in this series? If user2 never watches episodes in the series then episodes will never be deleted from the series.

minimum_episode_behavior allows a way to choose how the script will behave.

Based on looking at the definitions of those parameters I suspect I was wrong in my assumptions. And I'm not sure I understand. It almost looks as if its going to always center on a single user to determine which episodes to delete. So if user A favorites an episode, but based on Minimum_episode_behavior user B is the one it selects..... And if User B doesn't have that episode favorites, the episode would get deleted. Which is not what I want. I wanted to retain 14 days worth of items regardless of played status. Keep all favorites for any user. and I thought it would be nice to set a minimum to not drop below 10 episodes for a series (ie podcasts that are 5 days a week, wouldn't fall below 2 weeks).

I touched on it above. But each user is considered during the episode_control logic. Favorited media_items are removed from the delete list before episode_control logic is run.

One piece of information is gathered for each series:

• Total number of episodes on disk

Two pieces of information are found for each user:

• Number of watched episodes
• Number of unwatched episodes

From the three pieces of information above the script can determine how many episodes watched and unwatched are to be deleted and kept for each user following the values in minimum_episodes and minimum_played_episodes.

• If Max Played is chosen the user with highest number of watched episodes determines the behavior of episode_controls
• If Min Played is chosen the user with lowest number of watched episodes determines the behavior of episode_controls
• If Max Unplayed is chosen the user with highest number of unwatched episodes determines the behavior of episode_controls
• If Min Unplayed is chosen the user with lowest number of unwatched episodes determines the behavior of episode_controls
• If a user name or user Id is entered, this user determines the behavior of episode_controls

Maybe I cant get what I want with this parameter, and I should let it go?

If this is not working, I need to fix it. There are others using this feature as well. But until I can recreate the issue and fix it you may be better of skipping it for now.

[terrelsa13]

Try v5.8.22-beta with minimum_episodes: 10. Let me know how it works.

[terrelsa13]

I just looked at your updated docker instructions. nice. Can I use a version tag like [v5.8.21-beta] with your docker solution, or am i constrained to just "latest" and "dev"? Once i get it going I'd like to lock onto a version.

Realized I never gave an answer for this question. Yes there are versioned tags.

Image tags

• ghcr.io/terrelsa13/mumc:latest - The latest stable release. These install instructions do not apply for v5.7.3 and earlier.
• ghcr.io/terrelsa13/mumc:MUMC-Latest - The latest stable release. These install instructions do not apply for v5.7.3 and earlier.
• ghcr.io/terrelsa13/mumc:v#.#.# - A specific stable release. These install instructions do not apply for v5.7.3 and earlier.
• ghcr.io/terrelsa13/mumc:MUMC_dev - The latest beta release.
• ghcr.io/terrelsa13/mumc:v#.#.#-beta - A specific beta release.

[Doug411]

Thanks again. No worries about responsiveness. Your support of this is unparalleled and appreciated.

I'll give it a go. What do you think is the best choice for minimum_episode_behavior for my desired outcome? I'm having a difficult time wrapping my head around it.. I want to keep 10 episodes regardless of played status. Sounds like I don't have to care about favorites, because those are tossed out before episode control.

[Doug411]

FYI, I saw v5.8.24-beta was out there and more current than the version you suggested, so I checked that out. I'll give it a go now...

[Doug411]

Ok, just ran v5.8.24-beta. Definately progress. It returned some items in the final delete list. I haven't validated the list entirely. But its a subset of the items by individual, so it is directionally correct for sure. The perplexing part, is that I have 6 separate (podcast) series that are all blacktagged with the same tag. However, now, only 2 of the six series are showing in the top delete list by person. And therefore also in the summary delete after episode control.

[Doug411]

FYI , commented out episode control and re-ran with v5.824-beta and still only the same 2 of the 6 black-tagged series are suggested for deletion by person and in summary. So it doesn't seem the problem is with episode control, perhaps something else introduced in the newer versions.

[terrelsa13]

Set DEBUG: 1. Run the script again. Attach/Pastebin the mumc_DEBUG.log to the reply.

This will help me see what you see.

[terrelsa13]

Leave episode_control off. We want to get to the bottom of the missing blacktagged series before we start adding variables into the mix

[Doug411]

sent the link to the log to your Gmail (easier than scrubbing names).

[terrelsa13]

Ok. You mentioned there are x6 blacktagged series we are expecting to be shown in the results.

I see these x3 series below which contain the blacktags; Retain14d and/or auto-purge:

• Lex F.
• Tucker C.
• Bill M.

What are the other x3 we are not seeing?

The following steps will perform a manual search for your blacktags. Then we will try to find one of the missing media_items within the results returned from the search.

1. Manually enter the following URL into a browser that has access to the Emby/Jellyfin server.

   1. This will query the server for all items in the youtube library containing either of these blacktags: Retain14d and auto-purge.

      http://localhost:8096/Users/8503cbb714f84f709ea16fb61c2f9163/Items?ParentID=608e1ed1e97ee5c4556f3d0299ac9894&IncludeItemTypes=Episode,Season,Series,CollectionFolder&Fields=ParentId,Path,Tags,MediaSources,DateCreated,Genres,Studios,UserDataPlayCount,UserDataLastPlayedDate,SeriesStudio,seriesStatus&Recursive=True&SortBy=SeriesSortName,ParentIndexNumber,IndexNumber,Name&SortOrder=Ascending&EnableImages=False&CollapseBoxSetItems=False&Tags=Retain14d|auto-purge&EnableUserData=True&api_key=YOUR_API_KEY_HERE

2. Next open a new browser tab/window.

3. Log into Jellfyin.

4. In your head, choose x1 of the x3 missing shows we are expecting to see.

5. Navigate to that show in the Jellyfin GUI.

6. Our goal is to find the itemId for the media_item containing the blacktag in its metadata. (It is important we use the media_item that contains the blacktag(s))

   1. If the series has the blacktag in its metadata; navigate to the series and selected it to open it.
   2. If a season has the blacktag in its metadata; navigate to the season and selected it to open it.
   3. If an episode has the blacktag in its metadata; navigate to the episode and select the "i-in-a-circle" to the right of the episode to open it.

7. Next, look at URL bar at the top. It will look similar to the image below.

   1. The portion circled in green, between id= and the &, is the itemId for the media_item.

9. Copy the itemId.
10. Go back to the original tab/window with the results.
11. Do a search (ctrl+f) in the webpage and look for the itemId we just copied.
12. Is it there?

FYI - Tags are case sensitive. These are all treated as different tags.

• MyTag
• mytag
• myTag
• Mytag

[Doug411]

Yes, its in the query. I sent you an email with the content of the query and the id that I located in Jellyfin.

FYI, this is somewhat new. A few versions ago, I recall having all my blacktagged podcasts in the delete queue.

[terrelsa13]

That's strange. The series is found but the children are not fetched.

Let's simulate fetching the children of the Megyn K. show and see if seasons and/or episodes are returned.

Copy/Past this into a browser that has access to the Jellyfin server:

http://localhost:8096/Users/8503cbb714f84f709ea16fb61c2f9163/Items?ParentID=1bbe252b592e2386b28927bdd5c0e92c&IncludeItemTypes=&IsPlayed=&Fields=Id,Path,Tags,MediaSources,DateCreated,Genres,Studios,SeriesStudio,UserData&Recursive=True&SortBy=SeriesSortName,AlbumArtist,ParentIndexNumber,IndexNumber,Name&SortOrder=Ascending&CollapseBoxSetFalse&EnableImages=False&api_key=YOUR_API_KEY_HERE

[Doug411]

Returns episodes

[terrelsa13]

I am stumped. I must be missing something. You mind emailing me the results from the last query?