Open Doug411 opened 1 month ago
terrelsa13
commented
1 month ago Hey there!
Attach or send a pastebin link to your config file (please do not copy/paste). Then I will be able to see exactly what settings are there/not there and can help you get MUMC working the way you want it to.
I will be able to answer your other questions after work today.
Doug411
commented
1 month ago Thanks for the help. Much appreciated! see patebin link. I had tags enabled at one point, and got an error, so I just blacklisted the one folder. I was trying to get it working in its simplest form, before I ventured too far. I still would like clarity on tags and blacklists (do I need to blacklist if I blacktag at the item level? Can I black tag a folder, etc). Also unclear on how it works with multiple users. I read through the help but wasn't quite sure of the end result. (https://pastebin.com/SB7bqyy5)
terrelsa13
commented
1 month ago FYI - I do not see a pastebin link in your response.
Answers to your questions:
I also want to clear up my understanding of how the script works. If I use tags, do I still have to blacklist the library that I'm evaluating? In other words is it possible to NOT blacklist the podcast library, but blacktag items in a library that is whitelisted?
- It depends on how your configuration is setup.
- media_items in whitelisted libraries can be searched for blacktags and/or whitetags or neither.
- media_items in blacklisted libraries can be searched for blacktags and/or whitetags or neither.
What happens if I tag the individual podcast folders, versus tagging the items? Tubeachivist doesn't have functionality to tag items upon download. I'm trying to figure out the best way to accomplish a folder based retain/purge logic. ideas?
- Tagging a parent applies to all of its children.
- I see I don't explain this in the wiki; but the parent-child relationship applies to tags and favorites.
- Take a look at Blacktagging vs Whitetagging.
I also want to understand more why you are allowing multiple monitor users. If I monitor 5 users played status.... Is the main use case to not delete until all 5 have watched and it meets the days criteria? Wow, that's complex if so.
- Short answer; yes. MUMC allows several combinations of options to specify when a media_item should be deleted.
- Everyone wants to manage their media differently.
TLDR; Originally this script started out monitoring a single user; then the requests came to make it work for multiple users.
To keep things relatively simple there are behavioral_statements.
At a high level these allow a flexible way to do things like this:
advance_settings:
behavioral_statements:
episode:
conditional_behavior: favorite
action: delete
user_conditional: any
played_conditional: all_any
action_control: 3
dynamic_behavior: falseuser_conditional
played_conditional
action_control
action
advance_settings:
behavioral_statements:
episode:
conditional_behavior: favorite
action: delete
user_conditional: any
played_conditional: all_all
action_control: 5
dynamic_behavior: trueuser_conditional
played_conditional + (dynamic_behavior)
action_control
action
advance_settings:
behavioral_statements:
episode:
conditional_behavior: whitelist
action: keep
user_conditional: any
played_conditional: all_played
action_control: 5
dynamic_behavior: trueuser_conditional
played_conditional + (dynamic_behavior)
action_control
action
Also, what is the best way to potentially change the retention period at the podcast level? eg Podcast (Jellyfin Series Library) -Podcast 1 (Sub Folder) 14 days -Podcast 2 (Sub Folder) 21 days -Podcast 3 (Sub Folder) 30 days
There are two options that come to mind.
Set up x3 instances of MUMC (e.g. MUMC14, MUMC21, MUCM30)
Create three blacktags (e.g. "14 Day Delete", "21 Day Delete", "30 Day Delete")
In the MUMC14 instance setup the config as shown below:
basic_settings:
filter_statements:
episode:
created:
condition_days: 14
count_equality: '>='
count: 0
behavioral_control: true
advanced_settings:
filter_statements:
episode:
query_filter:
favorited: true
whitetagged: false
blacktagged: true
whitelisted: false
blacklisted: false
behavioral_statements:
episode:
favorited:
action: keep
user_conditional: any
played_conditional: ignore
action_control: 3
blacktagged:
action: delete
user_conditional: any
played_conditional: any_created
action_control: 5
dynamic_behavior: false
tags: ["14 Day Delete"]
delete_empty_folders:
episode:
season: true
series: trueRepeat for the MUMC21 and MUMC30 instances.
Set up x3 instances of MUMC (e.g. MUMC14, MUMC21, MUCM30)
Create x3 libraries
During setup of the MUMC14 instance only blacklist the "Podcast14 Library"
In the MUMC14 instance setup the config as shown below:
basic_settings:
filter_statements:
episode:
created:
condition_days: 14
count_equality: '>='
count: 0
behavioral_control: true
advanced_settings:
filter_statements:
episode:
query_filter:
favorited: true
whitetagged: false
blacktagged: false
whitelisted: false
blacklisted: true
behavioral_statements:
episode:
favorited:
action: keep
user_conditional: any
played_conditional: ignore
action_control: 3
blacklisted:
action: delete
user_conditional: any
played_conditional: any_created
action_control: 5
dynamic_behavior: false
delete_empty_folders:
episode:
season: true
series: trueRepeat for the MUMC21 and MUMC30 instances.
terrelsa13
commented
1 month ago To be clear, you only need to run x3 separate instances of MUMC if they are docker containers.
When running outside of a docker container, multiple config files can be created. And then use the -c command line option.
mumc_config14.yaml
mumc_config21.yaml
*mumc_config30.yaml
MUMC has Command Line Options which allow specifying a config file to pass into it.
/path/to/python /path/to/mumc.py -c /path/to/mumc_config14.yaml/path/to/python /path/to/mumc.py -c /path/to/mumc_config21.yaml/path/to/python /path/to/mumc.py -c /path/to/mumc_config30.yamlThanks. Very Helpful. Can you see my pastebin? It was in pasted into a codeblock before I think. I took it out and it appears as a link now.
In theory I understand everything you described. However, I still don't understand why my setup was only flagging watched content for deletion.
To simplify my setup to start, this is what I want to have happen. I would like to monitor 5 users for (favoriting action only). On the Youtube Library, I'd like to add 14 day delete tags to SOME of the subfolders (Podcast 1,2,3). Let's say I have also (Podcast 4,5,6), that for now I would add no tag to (or perhaps I need to add a whitetag?).
I want the content in folders 1,2,3 to be purged after 14 days unless it is in someone's favorites or the individual item is white tagged. I don't want to consider watched status.
If I can get some help on that (with my pastebin config), I should have no trouble following the example above to extend it for different retention periods if I desire. It looks like I'll end up going with your option 1, as the tubearchivist solution is pretty heavy (lots of ram for indexing and elastic search). It creates subfolders under 1 parent. Running multiple instances of tubearchivist to create multiple libraries would be a big load.....and alternatively I don't want each podcast to be its own individual library (i'd want them grouped logically in jellyfin). So I think the tagging example would work well (as long as I can just tag the podcast folder (below the library and above the content items as in the example below.
Video Podcast (Jellyfin library folder named 'Youtube' on Server)
-Podcast 1 (BLACKTAG)
-video 1
-video 2 (Favorite)
-video 3 (whitetag)
-Podcast 2 (BlACKTAG)
-video 1
-video 2
-Podcast 3 (BLACKTAG)
-video 1
-video 2
-Podcast 4 (no tag?)
-video 1
-video 2
-Podcast 5 (no tag?)
-video 1
-video 2
-Podcast 6 (no tag?)Yep! I can see the pastebin now. Much appreciated. I will re-read this tomorrow evening after work to see how I can help you.
terrelsa13
commented
1 month ago In general your request is possible. Just not exactly how you have described using Jellyfin.
Jellyfin does not allow sub-library-folder actions via the GUI (see this related Known Limitation). If you want to favorite or tag sub-library-folders you will either need to switch to Emby or write your own script to manipulate the API to tag the sub-library-folders (which I'm not 100% sure Jellyfin would let happen via the API either).
With Jellyfin tv shows and movies you can tag:
To simplify my setup to start, this is what I want to have happen. I would like to monitor 5 users for (favoriting action only). On the Youtube Library, I'd like to add 14 day delete tags to SOME of the subfolders (Podcast 1,2,3). Let's say I have also (Podcast 4,5,6), that for now I would add no tag to (or perhaps I need to add a whitetag?).
I want the content in folders 1,2,3 to be purged after 14 days unless it is in someone's favorites or the individual item is white tagged. I don't want to consider watched status.
I removed the admin_settings portion from my example config. The config is yaml, it is meant to be human readable and easy to understand. You should be able to copy/paste my example below into your config. It is ok to completely overwrite the basic_settings and advanced_settings sections. Just don't overwrite the admin_settings, version, or DEBUG section sections.
Doug411
commented
1 month ago I looked at your config and I understand it. I was missing logic in the behavioral statements. Developing a script that will work across both Jellyfin and Emby with the flexibility to handle multiple users and scenarios, is no easy task. Kudos. I'm iimpressed.
I'm not sure I fully understand the logic of action control 3 vs 5 for keep vs delete.... I'll have to play with it some. FYI.. I tried the docker and got an error after choosing people/libraries. I'll try the native version and let you know if it finishes.
Traceback (most recent call last): File "/usr/src/app/mumc_modules/mumc_config_import.py", line 93, in importConfig import mumc_config as cfg ModuleNotFoundError: No module named 'mumc_config'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/src/app/./mumc.py", line 147, in
Doug411
commented
1 month ago nevermind on the docker issue. I didn't have the empty file out there. working through some yaml issues....
Doug411
commented
1 month ago Is all of this required? I remember reading something about having changed to minimal config. I guess I can run through the setup again, and be even more surgical. I remember questionining whether I should leave in the filtering/favorites advanced pieces.
ConfigNameError: basic_settings > filter_statements > episode > played > condition_days is missing from mumc_config.yaml ConfigNameError: basic_settings > filter_statements > episode > played > count_equality is missing from mumc_config.yaml ConfigNameError: basic_settings > filter_statements > episode > played > count is missing from mumc_config.yaml ConfigNameError: advanced_settings > behavioral_statements > episode > favorited > advanced > genre is missing from mumc_config.yaml ConfigNameError: advanced_settings > behavioral_statements > episode > favorited > advanced > season_genre is missing from mumc_config.yaml ConfigNameError: advanced_settings > behavioral_statements > episode > favorited > advanced > series_genre is missing from mumc_config.yaml ConfigNameError: advanced_settings > behavioral_statements > episode > favorited > extra > library_genre is missing from mumc_config.yaml ConfigNameError: advanced_settings > behavioral_statements > episode > favorited > advanced > studio_network is missing from mumc_config.yaml ConfigNameError: advanced_settings > behavioral_statements > episode > favorited > advanced > studio_network_genre is missing from mumc_config.yaml ConfigNameError: advanced_settings > behavioral_statements > episode > whitetagged > tags is missing from mumc_config.yaml ConfigNameError: advanced_settings > behavioral_statements > episode > blacktagged > tags is missing from mumc_config.yaml ConfigNameError: advanced_settings > behavioral_statements > episode > whitelisted > action_days is missing from mumc_config.yaml ConfigNameError: advanced_settings > behavioral_statements > episode > whitelisted > user_conditional is missing from mumc_config.yaml ConfigNameError: advanced_settings > behavioral_statements > episode > whitelisted > played_conditional is missing from mumc_config.yaml ConfigNameError: advanced_settings > behavioral_statements > episode > blacklisted > action_days is missing from mumc_config.yaml ConfigNameError: advanced_settings > behavioral_statements > episode > blacklisted > user_conditional is missing from mumc_config.yaml ConfigNameError: advanced_settings > behavioral_statements > episode > blacklisted > played_conditional is missing from mumc_config.yaml
terrelsa13
commented
1 month ago What version are you using? The minimum config feature does not work until >= v5.8.0-beta. If using Docker check out this post from the last issue:
Doug411
commented
1 month ago Very strange. I was running it native when it generated those errors. Then I ran it in docker and it didn't generate any yaml in the advanced section at all (not even for the blacktags and white tags for which it prompted in the script. I pasted in your code and it ran first go. Its processing a lot of episodes. I think it runs for a bit and restarts. I tried going into the shell to try to find a log. However, it restarts before I can locate one. What's the best way to determine what's flagged for deletion with the docker version? One thought would be instead of mounting to the config.yaml, mount to a config folder. The script could look for the mmu_config.yaml in the config folder, and output the log to the same folder. The back to back restarts seem a bit problematic for diagnosis though. Not sure if it wipes the logs on each run or adds to them (as I haven't been able to get to a log). Thoughts?
Doug411
commented
1 month ago fyi, tried watching the log in portainer, but its moving through episodes too fast for me to analyze, and then restarts
terrelsa13
commented
1 month ago Very strange. I was running it native when it generated those errors.
Native version >= 5.8.0-beta? Or <= 5.7.3?
Then I ran it in docker and it didn't generate any yaml in the advanced section at all (not even for the blacktags and white tags for which it prompted in the script.
Ahhh good catch here. Script is not saving the config sections which are not longer set to default values during script creation.
I tried going into the shell to try to find a log.
The mumc_DEBUG.log should be in the same location as the mumc_config.yaml.
Create the mumc_DEBUG.log file outside of docker. In the same location as mumc_config.yaml.
Then add a new volume:
volumes:
- "/opt/MUMC/mumc_config.yaml:/usr/src/app/mumc_config.yaml"
- "/opt/MUMC/mumc_DEBUG.log:/usr/src/app/mumc_DEBUG.log"The back to back restarts seem a bit problematic for diagnosis though. Not sure if it wipes the logs on each run or adds to them (as I haven't been able to get to a log). Thoughts?
Yeah, it definitely deletes any existing mumc_DEBUG.log anytime MUMC runs. So my above suggestion will make the debug file accessible outside of the container. But it will be difficult to see the deletion summary before MUMC runs again.
One thought would be instead of mounting to the config.yaml, mount to a config folder. The script could look for the mmu_config.yaml in the config folder, and output the log to the same folder.
I considered this. But didn't have the time/patients to mess with the configuration import logic at the time.
Looks like there are still a few things needing to be worked out with the containerized script. I likely will not be able to make any good headway on this until the weekend.
terrelsa13
commented
1 month ago fyi, tried watching the log in portainer, but its moving through episodes too fast for me to analyze, and then restarts
I thought the portainer logs for a container could be paused? Even if it moves thru episodes quickly the portainer logging page has a buffer which allows scrolling back some number of lines.
terrelsa13
commented
1 month ago If the portainer log buffer doesn't go back far enough, try viewing the container logs in a terminal window.
docker logs -f mumcdocker logs --since 10m mumcdocker logs --since 10m mumc > /path/to/save_file.logi changed the restart policy to no and exported the logs. It processes through files for 4 of 6 users. It gets an error on user 5, stops and restarts again. The error is below. Even more dissappointing is that the filter isnt picking up any of the blacktags on the series folder. Answering your question from ealier today, the structure is Just Library, Series (No Season). I'm running the tubearchivist plugin for jellyfin. It is creating Season folders in the Jellyfin UI (but its not stored that way on disk). All episodes for a series (across all seasons) are in one Series folder along with associated metadata and subtitles. I added the blacktags to the series folder using the meta data manager in jellyfin. I could have sworn I saw some data flying by earlier today in portainer that looked like some blacktags were being assigned. But its definately not doing it now (i've run it many times, and have the cumulative logs)
2024-06-05T01:35:53.010147080Z Mitchell xxxx-xxxx - ca7ad35ea9xxxxxxxxxxxxxxxxxxxx 2024-06-05T01:35:53.010149250Z ---------------------------------------------------------------------------------------- 2024-06-05T01:35:53.010151509Z ---------------------------------------------------------------------------------------- 2024-06-05T01:35:53.010153739Z 2024-06-05T01:35:53.010155830Z HTTPError: Unable to get information from server during processing of: season_info_for_32d219223d3a18d2102c7a1f6d444a1e 2024-06-05T01:35:53.010158557Z Object: <bound method Request.header_items of <urllib.request.Request object at 0x7d66163b5ad0>> 2024-06-05T01:35:53.010161297Z URL: http://localhost:8096/Users/ca7ad35ea91c4a5ba26190d19e554563/Items/32d219223d3a18d2102c7a1f6d444a1e?enableImages=False&enableUserData=True&Fields=ParentId,Genres,Tags,RecursiveItemCount,ChildCount,Type 2024-06-05T01:35:53.010165960Z Method: 2024-06-05T01:35:53.010168346Z Header: {'Authorization': 'MediaBrowser Client="mumc.py", Device="Multi-User Media Cleaner", DeviceId="MUMC", Version="5.8.9-beta", Token="abcf86fab5a44978b95f502caf140e6b"', 'Content-type': 'application/json'} 2024-06-05T01:35:53.010171114Z Data: None 2024-06-05T01:35:53.010173308Z 2024-06-05T01:35:53.010175420Z HTTPError: 404 - Not Found 2024-06-05T01:35:53.010177684Z 2024-06-05T01:35:53.010179822Z Check password and/or remove any GUI API keys for MUMC 2024-06-05T01:50:32.616000519Z 2024-06-05T01:50:32.616025448Z ----------------------------------------------------------------------------------------
Doug411
commented
1 month ago Also, are tags User based, or if one user creates a tag is it there for all users? Just curious..... I did search on Blacktags across all the cumulative rows. I definately don't have any yes'. I was more curious of what to expect to see when its working.
Doug411
commented
1 month ago FYI. I was unable to mount the debug log the way indicated above. I tried creating the file, but I get an error message indicating device, resource busy. If the file isn't there, it creates a folder upon execution.
I kind of think the optimal way to run the docker solution would be to set the restart policy to no (as I have it). A restart can just be scheduled with cron (as you do the native solution). This would be perfect if the log file was viewable in the mounted volume. It would be just like native (without having to worry about dependencies)...
Now if I can just get it to recognize my blacktags.... :) help... :)
FYI... I may run it on my normal TV Shows as well. I need to look if that library has season folders. Is it going to be a problem if one library does and one doesn't have season folders?
Doug411
commented
1 month ago FYI. This is how the podcasts are organized in the GUI. I attempted to use two different tags as depicted (auto-purge and Retain14d). When MUMC runs it doesn't assign either tag to the underlying items. All episodes are marked as not being blacktagged. The tags are on the series item.
terrelsa13
commented
1 month ago i changed the restart policy to no and exported the logs. It processes through files for 4 of 6 users. It gets an error on user 5, stops and restarts again. The error is below. Even more dissappointing is that the filter isnt picking up any of the blacktags on the series folder. Answering your question from ealier today, the structure is Just Library, Series (No Season). I'm running the tubearchivist plugin for jellyfin. It is creating Season folders in the Jellyfin UI (but its not stored that way on disk). All episodes for a series (across all seasons) are in one Series folder along with associated metadata and subtitles. I added the blacktags to the series folder using the meta data manager in jellyfin. I could have sworn I saw some data flying by earlier today in portainer that looked like some blacktags were being assigned. But its definately not doing it now (i've run it many times, and have the cumulative logs)
2024-06-05T01:35:53.010147080Z Mitchell xxxx-xxxx - ca7ad35ea9xxxxxxxxxxxxxxxxxxxx 2024-06-05T01:35:53.010149250Z ---------------------------------------------------------------------------------------- 2024-06-05T01:35:53.010151509Z ---------------------------------------------------------------------------------------- 2024-06-05T01:35:53.010153739Z 2024-06-05T01:35:53.010155830Z HTTPError: Unable to get information from server during processing of: season_info_for_32d219223d3a18d2102c7a1f6d444a1e 2024-06-05T01:35:53.010158557Z Object: <bound method Request.header_items of <urllib.request.Request object at 0x7d66163b5ad0>> 2024-06-05T01:35:53.010161297Z URL: http://localhost:8096/Users/ca7ad35ea91c4a5ba26190d19e554563/Items/32d219223d3a18d2102c7a1f6d444a1e?enableImages=False&enableUserData=True&Fields=ParentId,Genres,Tags,RecursiveItemCount,ChildCount,Type 2024-06-05T01:35:53.010165960Z Method: 2024-06-05T01:35:53.010168346Z Header: {'Authorization': 'MediaBrowser Client="mumc.py", Device="Multi-User Media Cleaner", DeviceId="MUMC", Version="5.8.9-beta", Token="abcf86fab5a44978b95f502caf140e6b"', 'Content-type': 'application/json'} 2024-06-05T01:35:53.010171114Z Data: None 2024-06-05T01:35:53.010173308Z 2024-06-05T01:35:53.010175420Z HTTPError: 404 - Not Found 2024-06-05T01:35:53.010177684Z 2024-06-05T01:35:53.010179822Z Check password and/or remove any GUI API keys for MUMC 2024-06-05T01:50:32.616000519Z 2024-06-05T01:50:32.616025448Z ----------------------------------------------------------------------------------------
Ok, I will need your help getting a mumc_DEBUG.log during a run that is generating this error.
DEBUG: 4mumc_DEBUG.log to your reply OR pastebin it and reply with the link. (no direct copy/paste into the reply textbox please, it always messes up the formatting and makes it difficult to decipher what is happening.)I will also need a copy the mumc_config.yaml. Same thing, either attach to your reply or pastebin.
Also, are tags User based, or if one user creates a tag is it there for all users? Just curious..... I did search on Blacktags across all the cumulative rows. I definately don't have any yes'. I was more curious of what to expect to see when its working.
I kind of think the optimal way to run the docker solution would be to set the restart policy to no (as I have it). A restart can just be scheduled with cron (as you do the native solution). This would be perfect if the log file was viewable in the mounted volume. It would be just like native (without having to worry about dependencies)...
I agree with your suggestion. Truthfully did not know this was possible. Thanks for the feedback!
I need to look if that library has season folders. Is it going to be a problem if one library does and one doesn't have season folders?
I took a look at the tag logic. It should not matter if there is a season folder.
When a media_item is returned during the query:
season attribute.
parent attribute.season/parentseason/parent has tags that match what's in the configseason/parent is then check for its 'seriesorparentattribute and this repeats until we get to the top (aka thelibrary`)All episodes are marked as not being blacktagged. The tags are on the series item.
The debug logs I requested at the top of this reply will help me understand why MUMC is not considering the media_items tagged. But also make sure there is no rouge spaces, tabs, underscores, etc... Also if there are quotes around the tags in the config, try removing the quotes and see if that helps.
Doug411
commented
1 month ago Thanks so much for all your help. I don't know if the portainer logs are the same as the debug.log. I cant get to the debug.log file with the docker version, as it didn't run when I tried mounting it to an empty file. I'll try the native version. Looking back at your prior comment, seems I was running the 5.7.3 version. Should I clone a different version? 5.8.0-beta?
Doug411
commented
1 month ago config https://pastebin.com/iGYVQYC2
the debug log is 354MB, too big..... mostly because it is considering my regular shows folder. Is there a way to get it to run on a subset? It's too big for my chromebook to manipulate.
I shared the log to your gmail account listed on this site. You should get a link to the doc in drive. Hopefully you have a little more powerful pc to navigate it.
terrelsa13
commented
1 month ago Thanks so much for all your help.
Np! Been in your shoes.
I don't know if the portainer logs are the same as the debug.log.
Nope. They are different.
I'll try the native version. Looking back at your prior comment, seems I was running the 5.7.3 version. Should I clone a different version? 5.8.0-beta?
Yes, for now let's stick to the native version MUMC v5.8.9. This way we are not fighting Docker "unknowns" and can figure out the no-tags issues.
the debug log is 354MB, too big.....
My b. I should have mentioned it may become a big file.
mostly because it is considering my regular shows folder. Is there a way to get it to run on a subset? It's too big for my chromebook to manipulate.
Yeah we can disable all libraries for all users. Then enable the "youtube"/podcast library only for 1-2 users to keep it simple.
lib_enabled: true in the config file.lib_enabled: truelib_enabled: falselib_enabled: true just for the "youtube"/podcast library of the 1-2 users you've chosenDEBUG: 4mumc_DEBUG.log should be a more manageable sizeAs an example. This is what the "youtube"/podcast library entry in the config looks like. lib_enabled: true is the last variable for the library:
- lib_id: 608e1ed1e97ee5c4556f3d0299ac9894
collection_type: tvshows
path: /media/data/media/youtube
network_path: null
lib_enabled: trueYou want to set lib_enabled: false for all libraries. Then set lib_enabled: true only for the "youtube"/podcast library for 1-2 users.
I shared the log to your gmail account listed on this site. You should get a link to the doc in drive. Hopefully you have a little more powerful pc to navigate it.
Currently trying to open it. It's been spinning for 15mins. I'll let it keep trying for a little while longer. Probably better to try to do the steps I pointed out above; which should make a much smaller mumc_DEBUG.log file.
Doug411
commented
1 month ago will do. FYI, the big file opens when downloaded local. Hard to use 'find' function in it though. I'm using a chromebook with only 8gb ram, and it was really slow on finds (but it opened in about a minute or so,
Doug411
commented
1 month ago FYI, I sent another version with the suffix admin to your drive/Gmail. It's 50mb, so much smaller. It doesn't have the hard error in it, but should be useful in diagnosing the BLACKTAG issue. I'm going to try to generate the hard error now, with the user that errored out in the original post.
Doug411
commented
1 month ago sent another one with the hard error in it.
terrelsa13
commented
1 month ago Ok. Making some progress. I was able to figure out the tagging issue. I have a local v5.8.10-beta I have not pushed to Github yet, which has the fix for the missed tagging. Hoping to figure out this hard error before pushing it.
Now looking at the hard error. It appears to be happening when the script is it trying to get the "season04" folder information for the episode "Seinfeld - S04E01" for the Mitchell user.
Script is generating a HTTPError: 404 - Not Found error.
Here is what we are going to try:
user_iduser_idhttp://localhost:8096/Users/74d15d7a864441ccbabfc7122b9136af/Items/32d219223d3a18d2102c7a1f6d444a1e?enableImages=False&enableUserData=True&Fields=ParentId,Genres,Tags,RecursiveItemCount,ChildCount,Type&api_key=YOUR_API_KEY_HEREhttp://localhost:8096/Users/ca7ad35ea91c4a5ba26190d19e554563/Items/32d219223d3a18d2102c7a1f6d444a1e?enableImages=False&enableUserData=True&Fields=ParentId,Genres,Tags,RecursiveItemCount,ChildCount,Type&api_key=YOUR_API_KEY_HEREReturns for admin. Does not return for the Mitch user. I do have parental controls turned on for Mitch account. I will try turning it off and trying again......
{ "type": "https://tools.ietf.org/html/rfc9110#section-15.5.5", "title": "Not Found", "status": 404, "traceId": "00-1430f6134780f1c331c19f10d6194781-17a075021270fb37-00" }
OK, just redid it with parental controls turned off. It returns the same. The issue is with parental controls.
Doug411
commented
1 month ago was the tagging issue hard to fix?
terrelsa13
commented
1 month ago OK, just redid it with parental controls turned off. It returns the same. The issue is with parental controls.
Cool. Glad we figured this one out. But I want to make sure I understand what happened here.
was the tagging issue hard to fix? The tagging fix was not too bad. I did it to myself, in an earlier 5.8.x-beta I simplified the function that compares items. But when there were multiple items on either side of the comparison I somehow combined them into one big string instead of comparing them individually.
I'd say "normal" to fix. Unsurprisingly, I did it to myself. It was broken for all things needing comparison; not just tags.
Example: The issue was happening when either the config file or media item had multiple tabs.
"someTag", "myTag""yourTag", "myTag", "theirTag"Instead of starting with the first tag of the media_item and comparing it to each tag in the config file. Then repeating the same with the next media_item tag until a match is found (or not found). It was combining anything with multiple items for comparison into a single string. So "someTagmyTag" was being compared to "yourTagmyTagtheirTag".
I have also been working on these docker feature improvements you suggested.
These are what are taking longest time. Plus I do my best to test the script to make sure I didn't break something else before releasing it.
Good news though. I just released v.5.8.10-beta. I would try it native first, just in-case we run into any more issues, or I messed something up. Then once you are configured the way you like you could give the docker path a try.
Doug411
commented
1 month ago Thanks much. I cant imagine what the conditional logic must look like..... When I look at the wiki, it seems the flexibility is amazing.... which seems like it would translate to very complex code.
Yes, how you described the issue with Mitchell was mostly correct. It occurred on the Shows Library (not Youtube). He had access to Shows, but the parental controls precluded him from viewing the Seinfeld episode. I'd like to leave parental controls on (which would limit his access to that episode), but without it throwing an error. As of the last code base this was not possible.
I'll give the v5.8.10-beta another try now. What is the best way to switch between versions. I'm not a git expert. I've been using the checkout function. Is there a different way?
Doug411
commented
1 month ago Cool mods re: docker. I'm curious what you are going to allow for command line environment runtime variables.
Now that Im going to run it with restart= no.... I don't mind about having multiple docker containers (with different config files.... to control different retention periods. I can just use chron to schedule them to run once a day a half hour apart.
I thought about a potential enhancement to have some 'preset tags' (eg 'RetainXd' (where x could be an integer of days) for such purpose, as I would imagine some folks would find it valuable to have different retention periods on movies, shows, or even on specific series. But i'm personally not bothered by having a few containers for that purpose. But was just an idea of a potential future enhancement.
Doug411
commented
1 month ago When i do a git tag -list on the repository the last two I see are v5.8.2-beta v5.8.3-beta
I think i have v-5.8.3 checked out already. Whats weird is my config file says its v5.8.9-beta.
Are your changes in v5.8.3-beta, or do i need to change branches or something?
Doug411
commented
1 month ago ahh.. found it... I misread it thinking you were saying v5.8.1-beta... V5.8.10-beta in dev branch. I'll switch over to that
Doug411
commented
1 month ago Cant figure how to get to v5.810-beta. When I list its still showing me up to v5.8.3-beta. I'm in the dev branch. I need some rudimentry git help.
terrelsa13
commented
1 month ago Cool mods re: docker. I'm curious what you are going to allow for command line environment runtime variables.
I thought about a potential enhancement to have some 'preset tags' (eg 'RetainXd' (where x could be an integer of days) for such purpose, as I would imagine some folks would find it valuable to have different retention periods on movies, shows, or even on specific series. But i'm personally not bothered by having a few containers for that purpose. But was just an idea of a potential future enhancement.
Good idea here! If I were to do something like this this generic tag would need to contain the same information a basic filter statement contains: _condition_days, countequality, and count and way to indicate if it is for played or created filtering. To match the current functionality.
played10>=_1
created365==_0
It would not allow differing user specific retentions. But it would allow differing series, season, even episode specific retentions.
Cant figure how to get to v5.810-beta. When I list its still showing me up to v5.8.3-beta. I'm in the dev branch. I need some rudimentry git help.
There are too many ways to use GitHub. I change branches via VSCode, but that is because VSCode is the editor I use for coding/maintaining this script.
The easiest way would be to go to releases and look for the highest 5.8.#-beta version. At the bottom of each release section there is an Assets drop down. Expand it. Inside there will be a MUMC.zip and a MUMC.tar.gz. Download whichever you can "unzip". Once downloaded, it can be "unzipped" and inside will be a MUMC folder with everything needed to run that version.
I found some things not working as expected. Latest version is now v.5.8.14-beta.
Doug411
commented
1 month ago Ill check out 5.8.14-beta in the morning. Love your idea for tagging. Will be more user friendly for those not used to editing the yaml, and gives a lot of flexibility to taylor retention by content. If I made a tag created_14>=_0 I'd have my current logic, would I not? I was a bit confused on that or created14==-1 (to disable the play count all together). I know its not there yet... More asking so that I undertand how to edit the yaml in the short term.
Doug411
commented
1 month ago As I think about it... the tagging solution almost negates the whole need to edit the yaml entirely, doesn't it? The library to person monitoring would be done via the script. Blacklisting libraries is done via the script. Then the rest is just tags.. ( advanced users would obviously update the yaml directly if they want to change configuration, versus re-running the script... Its going to be faster...but those less comfortable would just rerun the script and retag if necessary)
Doug411
commented
1 month ago FYI, I just looked at the metadata manager in Jellyfin, and metadata/tags can be assigned at the library level. You could consider making lib_enabled: a standard tag as well.
Doug411
commented
1 month ago oh wait... was lib_enabled by person (I cant remember) If so , nevermind. Makes sense the way you are doing it. I assume you just check if anyone's monitoring, if nobody is monitoring you skip. I like the usability of being able to flip it off in metadata for everyone...but is a bit redundant with the checks you have to do anyway at the person level..
terrelsa13
commented
1 month ago If I made a tag created_14>=0 I'd have my current logic, would I not? I was a bit confused on that or created_14==-1 (to disable the play count all together). I know its not there yet... More asking so that I undertand how to edit the yaml in the short term.
Think of it like like this. Scripts looks at one media item at a time. Then asks the media_item these questions:
If both are true then the created filter statement for this media_item has been met.
created:
condition_days: 14
count_equality: '>='
count: 0For this generic tag thing it would be the same premise. Except it would ask those same questions again, but instead use the data from the generic tag.
Yeah, I still have to double check the -1 thing when I see it. Script was allowing -1 in the count, which was a bug. -1 should only allowed in the condition_days to mean disabled. Because a media_item cannot be played -1 day ago and it cannot be played -1 time.
Will be more user friendly for those not used to editing the yaml, and gives a lot of flexibility to taylor retention by content.
The yaml would still need to be edited. I am not aware of a way to send a wildcard query to the emby/jellyfin server. Just like today, script would need to know exactly what tags it should be asking the server to look for.
If I made a tag created_14>=_0 I'd have my current logic, would I not? I was a bit confused on that or created14==-1 (to disable the play count all together). I know its not there yet... More asking so that I undertand how to edit the yaml in the short term.
At this moment of theorizing about how it would work, you are 100% correct.
I don't want to get your hopes up about this though.
FYI, I just looked at the metadata manager in Jellyfin, and metadata/tags can be assigned at the library level. You could consider making lib_enabled: a standard tag as well.
Not sure I follow. But I will give a shot at where I think you are going. You can correct me if I am off base.
Tags already work from the child all the way up to the library parent. If the library were tagged with created_14>=_0 then every media_item in that library would be considered for that tag.
oh wait... was lib_enabled by person (I cant remember)
Yep. In the config each library for individual users can be enabled/disabled with lib_enabled:
If a library is disabled for a specific user, the other users with the library enabled are still considered when the script runs.
I like the usability of being able to flip it off in metadata for everyone...but is a bit redundant with the checks you have to do anyway at the person level.
The current solution for this is the configuration updater which basically re-runs a sub-section of the steps used to create the configuration. This can be enabled in the config or via the -u command line argument.
The way I would implement this would be to check which libraries a user is still allowed to access in the GUI. Which would allow enabling/disabling libraries in the GUI. I recognize Jellyfin does not allow changing sub-folder access. But Emby does and eventually Jellyfin should implement this feature. When that happens the "lib_enabled" tag functionality would be obsoleted.
Doug411
commented
1 month ago FYI. I tried the beta and get the following error. No worries about getting my hopes up on the tagging. I'll be more than happy to use it as is... I didn't realize you had a configuration updater routine. nice.
get_server_version - url request: <urllib.request.Request object at 0x7301df063df0> Response code: 200 Data Returned From The get_server_version Request: { "OperatingSystemDisplayName": "", "HasPendingRestart": false, "IsShuttingDown": false, "SupportsLibraryMonitor": true, "WebSocketPortNumber": 8096, "CompletedInstallations": [], "CanSelfRestart": true, "CanLaunchWebBrowser": false, "ProgramDataPath": "/config", "WebPath": "/jellyfin/jellyfin-web", "ItemsByNamePath": "/config/metadata", "CachePath": "/cache", "LogPath": "/config/log", "InternalMetadataPath": "/config/metadata", "TranscodingTempPath": "/config/transcodes", "CastReceiverApplications": [ { "Id": "F007D354", "Name": "Stable" }, { "Id": "6F511C87", "Name": "Unstable" } ], "HasUpdateAvailable": false, "EncoderLocation": "System", "SystemArchitecture": "X64", "LocalAddress": "http://192.168.1.200", "ServerName": "appserver-jellyfin", "Version": "10.9.6", "OperatingSystem": "", "Id": "8b224a34beca4c93ab27f20536c9e9ca" }
Processing EPISODE Items For UserId: 74d15d7a864441ccbabfc7122b9136af
No favorited media items are blacklisted
No favorited media items are whitelisted
No whitetagged media items are blacklisted
No whitetagged media items are whitelisted
No blacktagged media items are blacklisted
No blacktagged media items are whitelisted
No watched media items are blacklisted
No watched media items are whitelisted
No favorited media items are blacklisted
No favorited media items are whitelisted
No whitetagged media items are blacklisted
No whitetagged media items are whitelisted
No blacktagged media items are blacklisted
No blacktagged media items are whitelisted
No watched media items are blacklisted
No watched media items are whitelisted IsPlayed IsCreated Filter Value: playedAndUnplayed/n
No favorited media items are blacklisted
No favorited media items are whitelisted
No whitetagged media items are blacklisted
No whitetagged media items are whitelisted
No blacktagged media items are blacklisted
episode_blacktagged_from whitelisted_media_items - url request: <urllib.request.Request object at 0x7301df0f1270> Response code: 200 Data Returned From The episode_blacktagged_from whitelisted_media_items Request: { "Items": [ { "Name": "PBD Podcast", "ServerId": "8b224a34beca4c93ab27f20536c9e9ca", "Id": "e92f4a5907e7ed46c7939028095faefc", "DateCreated": "2024-06-03T23:00:37.596Z", "Path": "/media/data/media/youtube/UCGX7nGXpz-CmO_Arg-cgJ7A", "ChannelId": null, "Genres": [], "IsFolder": true, "ParentId": "badcda279c75b9a06c351cd3d5592fb8", "Type": "Series", "Studios": [ { "Name": "PBD Podcast", "Id": "6229f4adb0b6f8367b019e68697309c9" } ], "GenreItems": [], "UserData": { "UnplayedItemCount": 18, "PlaybackPositionTicks": 0, "PlayCount": 0, "IsFavorite": false, "Played": false, "Key": "e92f4a59-07e7-ed46-c793-9028095faefc" }, "AirTime": "", "AirDays": [], "Tags": [ "auto-purge", "Patrick Bet-David PBD Valuetainment podcast PBD podcast Patrick Bet-David Podcast Valuetainment Podcast Patrick Patrick Bet David Economy Politics Money Business", "Placeholder", "Retain14d" ], "DisplayOrder": "", "ImageBlurHashes": {}, "LocationType": "FileSystem", "MediaType": "Unknown" } ], "TotalRecordCount": 6, "StartIndex": 0 }
API Query Control Data For The NEXT LOOP: episode_blacktagged_from whitelisted_media_items Starting at record index: 1 Asking for 5 records Total records for this query is: 6 Are there records remaining: True IsPlayed IsCreated Filter Value: playedAndUnplayed/n
Find_Child_Of_Blacktagged_Item_From_Whitelist - url request: <urllib.request.Request object at 0x7301df0f18a0> Response code: 200 Data Returned From The Find_Child_Of_Blacktagged_Item_From_Whitelist Request: { "Items": [ { "Name": "Season 2024", "ServerId": "8b224a34beca4c93ab27f20536c9e9ca", "Id": "d05f3737b6d8b348b5632bdf87264727", "DateCreated": "2024-06-04T16:47:57.5512655Z", "PremiereDate": "2024-04-06T00:00:00.0000000Z", "ChannelId": null, "Genres": [], "ProductionYear": 2024, "IndexNumber": 2024, "IsFolder": true, "Type": "Season", "Studios": [], "GenreItems": [], "UserData": { "UnplayedItemCount": 18, "PlaybackPositionTicks": 0, "PlayCount": 0, "IsFavorite": false, "Played": false, "Key": "e92f4a59-07e7-ed46-c793-9028095faefc2024" }, "SeriesName": "PBD Podcast", "SeriesId": "e92f4a5907e7ed46c7939028095faefc", "Tags": [ "Retain14d", "auto-purge", "Placeholder" ], "PrimaryImageAspectRatio": 1, "SeriesPrimaryImageTag": "63c5075083702453f72087d92560ca6f", "ImageBlurHashes": { "Primary": { "63c5075083702453f72087d92560ca6f": "eTF|r=3YIEEROr;9BV%3f,NaAWwIsRoJn%T1Wrr=fkodRQNd%2t7R*" } }, "SeriesStudio": "PBD Podcast", "LocationType": "Virtual", "MediaType": "Unknown" } ], "TotalRecordCount": 20, "StartIndex": 0 }
API Query Control Data For The NEXT LOOP: Find_Child_Of_Blacktagged_Item_From_Whitelist Starting at record index: 1 Asking for 19 records Total records for this query is: 20 Are there records remaining: True
terrelsa13
commented
1 month ago You sure the error was copy/pasted? I am not seeing it.
terrelsa13
commented
1 month ago Unless this is what you meant:
Response code: 200If yes, codes from 200 - 299 mean everything is working as expected.
Try setting DEBUG: 0. That should hide all of the extra stuff happening in the background.
If it does not, that means I broke something.
Doug411
commented
1 month ago Below is what it shows in the terminal output. It stops within 2 seconds of starting. Doesn't appear to process any episodes. I have blacktags on two series. From the earlier log I sent it seems to get the first one, but not the second.
Traceback (most recent call last):
File "/opt/mumc/./mumc.py", line 147, in
Doug411
commented
1 month ago FYI, I have it turned on, for all users on both the shows, and YouTube libraries. It should have a bunch of episodes in the log. Its stopping on the keyerror and processing nothing. Execution time is about 1 second.
version: 5.8.9-beta basic_settings: filter_statements: episode: created:
condition_days: 14
count_equality: '>='
count: 0
behavioral_control: trueadvanced_settings: filter_statements: episode: query_filter:
#query whitelists for blacktagged media_items
favorited: false
whitetagged: false
blacktagged: true
whitelisted: true
blacklisted: falsebehavioral_statements: episode:
#no action if not favorited
favorited:
action: keep
user_conditional: any
played_conditional: ignore
action_control: 3
#keep whitetagged media_items
#no action if not whitetagged
whitetagged:
action: keep
user_conditional: all
played_conditional: ignore
action_control: 3
#delete blacktagged media_items meeting the basic_settings > filter_statement > episode > created query
#keep blacktagged media_items not meeting the basic_settings > filter_statement > episode > created query
blacktagged:
action: delete
user_conditional: all
played_conditional: any_created
action_control: 5
#ignore making delete or keep decisions based on the media_item being whitelisted
whitelisted:
action_control: 0
#ignore making delete or keep decisions based on the media_item being blacklisted
blacklisted:
action_control: 0whitetags:
Doug411
commented
1 month ago Also was looking at the stats at the beginning of the log. I don't see lines for Blagtagged items that are favorites. I definately have a few of those (but not under the admin user... They would be under the next user 'Doug'. Do you not write out that statistic in the log?
terrelsa13
commented
1 month ago Ahh ok. Go ahead and set DEBUG: 4. Run the script again. Then attach or pastebin the DEBUG log to the reply. That way I can see what is happening before the error stops the script.
Also attach or pastebin the config so I can try to reproduce it on my end.
terrelsa13
commented
1 month ago Very important. Please remember not to copy/paste things like the config, debug logs, errors, etc... straight into the comment text box.
I am not asking to be difficult. It truly makes it more difficult or impossible to find the issue on my side. Yaml for instance requires specific intention/spacing to work properly. When it is copy/pasted into the comment text box it looses or alters the the formatting. I then I have no way to know if the script is the issue or if the config file formatting is the issue.
It is ok to copy/paste into the comment box if a code block...
like this will be used.
But the proper formatting still needs to be verified when using a code block.It is easier to copy/paste into a text editor and attach the file. Or copy/paste into pastebin and send the link.
Hi again! I have a folder that I am populating with podcasts. I'm using TubeArchivist to download them to a Podcast Library. Each individual podcast is a series folder on Jellyfin. Many of the podcasts are daily. I'd like to set a retention period for 14 days, and delete old content regardless of whether it has been watched or not (unless its favorited), I've tried various incarnations, but the only items it seems to put in my delete queue are items that were watched by the monitored user.
This is what I thought would work based on my understanding of the logic. (turn off played evaluation, set my created condition, and put the count <0). However, it only flagged the played items.
episode: played: condition_days: -1 count_equality: '>=' count: 1 created: condition_days: 14 count_equality: '>=' count: -1 behavioral_control: true
Also, what is the best way to potentially change the retention period at the podcast level? eg
Podcast (Jellyfin Series Library) -Podcast 1 (Sub Folder) 14 days -Podcast 2 (Sub Folder) 21 days -Podcast 3 (Sub Folder) 30 days
I also want to clear up my understanding of how the script works. If I use tags, do I still have to blacklist the library that I'm evaluating? In other words is it possible to NOT blacklist the podcast library, but blacktag items in a library that is whitelisted?
What happens if I tag the individual podcast folders, versus tagging the items? Tubeachivist doesn't have functionality to tag items upon download. I'm trying to figure out the best way to accomplish a folder based retain/purge logic. ideas?
I also want to understand more why you are allowing multiple monitor users. If I monitor 5 users played status.... Is the main use case to not delete until all 5 have watched and it meets the days criteria? Wow, that's complex if so.