search

Schaka / janitorr

Cleans your Radarr, Sonarr, Jellyseerr and Jellyfin before you run out of space
GNU General Public License v3.0
313 stars 7 forks source link

Not doing anything? #22

Closed nothing2obvi closed 7 months ago

nothing2obvi commented 7 months ago

I am running v1.0.5. I set up a series with a tag of "14d" (I just tagged it, I didn't make a custom format or release profile or anything like that), set up a 14d tag in the config with an expiration of 14d, turned off dry-run, turned off media deletion, left tag deletion enabled, and started janitorr. The series has files grabbed more than 14 days ago. This is all I get:

03/21/2024
08:02:47 PM
2024-03-22T03:02:47.843Z  INFO 1 --- [           main] c.g.s.janitorr.JanitorrApplicationKt     : Starting JanitorrApplicationKt using Java 21.0.2 with PID 1 (/app/classes started by root in /)
03/21/2024
08:02:47 PM
2024-03-22T03:02:47.845Z  INFO 1 --- [           main] c.g.s.janitorr.JanitorrApplicationKt     : No active profile set, falling back to 1 default profile: "default"
03/21/2024
08:02:50 PM
2024-03-22T03:02:50.033Z  INFO 1 --- [           main] o.s.b.w.embedded.tomcat.TomcatWebServer  : Tomcat initialized with port 8978 (http)
03/21/2024
08:02:50 PM
2024-03-22T03:02:50.047Z  INFO 1 --- [           main] o.apache.catalina.core.StandardService   : Starting service [Tomcat]
03/21/2024
08:02:50 PM
2024-03-22T03:02:50.047Z  INFO 1 --- [           main] o.apache.catalina.core.StandardEngine    : Starting Servlet engine: [Apache Tomcat/10.1.18]
03/21/2024
08:02:50 PM
2024-03-22T03:02:50.130Z  INFO 1 --- [           main] o.a.c.c.C.[Tomcat].[localhost].[/]       : Initializing Spring embedded WebApplicationContext
03/21/2024
08:02:50 PM
2024-03-22T03:02:50.135Z  INFO 1 --- [           main] w.s.c.ServletWebServerApplicationContext : Root WebApplicationContext: initialization completed in 2118 ms
03/21/2024
08:02:50 PM
Standard Commons Logging discovery in action with spring-jcl: please remove commons-logging.jar from classpath in order to avoid potential conflicts
03/21/2024
08:02:52 PM
2024-03-22T03:02:52.419Z  INFO 1 --- [           main] o.s.b.w.embedded.tomcat.TomcatWebServer  : Tomcat started on port 8978 (http) with context path ''
03/21/2024
08:02:52 PM
2024-03-22T03:02:52.441Z  INFO 1 --- [           main] c.g.s.janitorr.JanitorrApplicationKt     : Started JanitorrApplicationKt in 5.182 seconds (process running for 6.051)

Here is the application.yml.

server:
  port: 8978

# File system access (same mapping as Sonarr, Radarr and Jellyfin) is required to delete TV shows by season and create "Leaving Soon" collections in Jellyfin
# Currently, Jellyfin does not support an easy way to add only a few seasons or movies to a collection, we need access to temporary symlinks
# Additionally, checks to prevent deletion on currently still seeding media currently require file system access as well
file-system:
  access: true
  leaving-soon-dir: "/media/jellyfin/Leaving Soon" # A directory this container can write to and Jellyfin can find under the same path - this will contain new folders with symlinks to files for Jellyfin's "Leaving Soon" collections
  from-scratch: true # Clean up entire directory and rebuild from scratch - this can help with clearing orphaned data - turning this off can save resources (less writes to drive)

application:
  dry-run: false
  leaving-soon: 14d # 14 days before a movie is deleted, it gets added to a "Leaving Soon" type collection (i.e. movies that are 76 to 89 days old)
  exclusion-tag: "janitorr_keep" # Set this tag to your movies or TV shows in the *arrs to exclude media from being cleaned up

  media-deletion:
    enabled: false
    movie-expiration:
      # Percentage of free disk space to expiration time - if the highest given number is not reached, nothing will be deleted
      # If filesystem access is not given, disk percentage can't be determined. As a result, Janitorr will always choose the largest expiration time.
      5: 15d # 15 days
      10: 30d # 1 month - if a movie's files on your system are older than this, they will be deleted
      15: 30d # 2 months
      20: 90d # 3 months
    season-expiration:
      5: 15d # 15 days
      10: 20d # 20 days - if a season's files on your system are older than this, they will be deleted
      15: 60d # 2 months
      20: 120d # 4 months

  tag-based-deletion:
    enabled: true
    minimum-free-disk-percent: 50
    schedules:
      - tag: “1d”
        expiration: 1d
      - tag: "14d"
        expiration: 14d
      - tag: “30d”
        expiration: 30d
      - tag: “60d”
        expiration: 60d
      - tag: “90d”
        expiration: 90d

clients:
  sonarr:
    enabled: true
    url: "http://10.0.0.224:8989"
    api-key: "keyhere"
  radarr:
    enabled: true
    url: "http://10.0.0.224:7878"
    api-key: "keyhere"
  ## You can only choose one out of Jellyfin or Emby.
  jellyfin:
    enabled: true
    url: "http://10.0.0.224:8096"
    api-key: "keyhere"
    username: janitorr
    password: pwhere
  jellyseerr:
    enabled: true
    url: "http://10.0.0.224:5055"
    api-key: "keyhere"

Am I forgetting to do something?

Schaka commented 7 months ago

This seems correct. Are you below 50% disk space?

If not, it won't do anything. I suggest enabling dry run and setting the threshold to 100. You should then see what it intends to do in the log.

If it still doesn't do anything at all, we can investigate further.

nothing2obvi commented 7 months ago

Thanks for your reply. I did as you suggested and I now get this, multiple times:

03/21/2024
10:53:37 PM
2024-03-22T05:53:37.457Z  INFO 1 --- [   scheduling-1] c.g.s.j.cleanup.AbstractCleanupSchedule  : Free disk space: 65.77629408490317%
03/21/2024
10:53:37 PM
2024-03-22T05:53:37.457Z  INFO 1 --- [   scheduling-1] c.g.s.j.cleanup.AbstractCleanupSchedule  : Not deleting Shows because minimum disk threshold was not reached.
03/21/2024
10:53:37 PM
2024-03-22T05:53:37.457Z  INFO 1 --- [   scheduling-1] c.g.s.j.cleanup.AbstractCleanupSchedule  : Free disk space: 65.77629408490317%
03/21/2024
10:53:37 PM
2024-03-22T05:53:37.458Z  INFO 1 --- [   scheduling-1] c.g.s.j.cleanup.AbstractCleanupSchedule  : Not deleting Movies because minimum disk threshold was not reached.
03/21/2024
10:53:37 PM
2024-03-22T05:53:37.458Z  INFO 1 --- [   scheduling-1] c.g.s.j.cleanup.AbstractCleanupSchedule  : Free disk space: 65.7762417992386%
03/21/2024
10:53:37 PM
2024-03-22T05:53:37.458Z  INFO 1 --- [   scheduling-1] c.g.s.j.cleanup.AbstractCleanupSchedule  : Not deleting Shows because minimum disk threshold was not reached.
03/21/2024
10:53:37 PM
2024-03-22T05:53:37.458Z  INFO 1 --- [   scheduling-1] c.g.s.j.cleanup.AbstractCleanupSchedule  : Free disk space: 65.77620912069825%
03/21/2024
10:53:37 PM
2024-03-22T05:53:37.458Z  INFO 1 --- [   scheduling-1] c.g.s.j.cleanup.AbstractCleanupSchedule  : Not deleting Movies because minimum disk threshold

Does it matter that while Sonarr added the episodes to the download client (on a remote seedbox), they weren't actually downloaded to my home server by Sonarr? I instead use FTPGrab to download them and use Hazel to sort them into their appropriate media folders.

On a similar note, will janitorr delete episodes and movies that Sonarr and Radarr didn't grab (they were instead added to the download client, downloaded, and sorted into their appropriate media folders all manually)?

Schaka commented 7 months ago

Janitorr will get the download times from your history in Sonarr/Radarr. Whatever is the first grab ever (if the quality profile doesn't allow for upgrades) or the most recent one (if it does allow for upgrades) is used to determine age.

It sends a delete request to Sonarr/Radarr and then to Jellyfin/Jellyseerr.

If the files are on a different system, it won't delete anything. Similarly, it won't touch anything that is in your library but not in Sonarr/Radarr.

However, if you do have Sonarr/Radarr, it DOES have media and you manually replaced it (inside the respective folders) without even using the import function, it will still treat whatever files you have as the to-be-deleted ones, no matter what time you actually replaced the folder content because all it has to go by is the history. You can look at the history in the Sonarr Web UI yourself.

For your case, it would use Sonarr/Radarr for the history and then use Jellyfin to delete the content.

You can try v1.0.6 - I added some more logging as well.

nothing2obvi commented 7 months ago

Thank you for your detailed reply. That's very helpful information, thank you.

I upgraded to v1.0.6 and I'm still getting logs exactly the same as my original post, with the disk space parameter set to 100 and dry-run enabled.

Schaka commented 7 months ago

I'm so confused. The condition is simple

getFreeSpacePercentage() <= applicationProperties.tagBasedDeletion.minimumFreeDiskPercent

If you're under 100% free disk space (i.e. always), it should absolutely not do any magic and just run the delete schedule. Can you enable debug or trace logging?

nothing2obvi commented 7 months ago

Sure, I enabled debug logging.

03/22/2024
02:24:39 AM
2024-03-22T09:24:39.967Z  INFO 1 --- [           main] c.g.s.janitorr.JanitorrApplicationKt     : Starting JanitorrApplicationKt using Java 21.0.2 with PID 1 (/app/classes started by root in /)
03/22/2024
02:24:39 AM
2024-03-22T09:24:39.972Z DEBUG 1 --- [           main] c.g.s.janitorr.JanitorrApplicationKt     : Running with Spring Boot v3.2.2, Spring v6.1.3
03/22/2024
02:24:39 AM
2024-03-22T09:24:39.973Z  INFO 1 --- [           main] c.g.s.janitorr.JanitorrApplicationKt     : No active profile set, falling back to 1 default profile: "default"
03/22/2024
02:24:45 AM
2024-03-22T09:24:45.280Z  INFO 1 --- [           main] o.s.b.w.embedded.tomcat.TomcatWebServer  : Tomcat initialized with port 8978 (http)
03/22/2024
02:24:45 AM
2024-03-22T09:24:45.356Z  INFO 1 --- [           main] o.apache.catalina.core.StandardService   : Starting service [Tomcat]
03/22/2024
02:24:45 AM
2024-03-22T09:24:45.357Z  INFO 1 --- [           main] o.apache.catalina.core.StandardEngine    : Starting Servlet engine: [Apache Tomcat/10.1.18]
03/22/2024
02:24:45 AM
2024-03-22T09:24:45.710Z  INFO 1 --- [           main] o.a.c.c.C.[Tomcat].[localhost].[/]       : Initializing Spring embedded WebApplicationContext
03/22/2024
02:24:45 AM
2024-03-22T09:24:45.714Z  INFO 1 --- [           main] w.s.c.ServletWebServerApplicationContext : Root WebApplicationContext: initialization completed in 5458 ms
03/22/2024
02:24:45 AM
Standard Commons Logging discovery in action with spring-jcl: please remove commons-logging.jar from classpath in order to avoid potential conflicts
03/22/2024
02:24:50 AM
2024-03-22T09:24:50.202Z  INFO 1 --- [           main] o.s.b.w.embedded.tomcat.TomcatWebServer  : Tomcat started on port 8978 (http) with context path ''
03/22/2024
02:24:50 AM
2024-03-22T09:24:50.252Z  INFO 1 --- [           main] c.g.s.janitorr.JanitorrApplicationKt     : Started JanitorrApplicationKt in 13.46 seconds (process running for 15.491)
03/22/2024
02:24:50 AM
2024-03-22T09:24:50.321Z DEBUG 1 --- [   scheduling-1] c.g.s.j.cleanup.TagBasedCleanupSchedule  : Deleting TV shows and movies with tag: TagDeleteSchedule(tag=“1d”, expiration=PT24H)
03/22/2024
02:26:12 AM
2024-03-22T09:26:12.846Z DEBUG 1 --- [   scheduling-1] c.g.s.j.cleanup.TagBasedCleanupSchedule  : Deleting TV shows and movies with tag: TagDeleteSchedule(tag=14d, expiration=PT336H)
03/22/2024
02:26:21 AM
2024-03-22T09:26:21.637Z DEBUG 1 --- [   scheduling-1] c.g.s.j.cleanup.TagBasedCleanupSchedule  : Deleting TV shows and movies with tag: TagDeleteSchedule(tag=“30d”, expiration=PT720H)
03/22/2024
02:26:29 AM
2024-03-22T09:26:29.544Z DEBUG 1 --- [   scheduling-1] c.g.s.j.cleanup.TagBasedCleanupSchedule  : Deleting TV shows and movies with tag: TagDeleteSchedule(tag=“60d”, expiration=PT1440H)
03/22/2024
02:26:41 AM
2024-03-22T09:26:41.151Z DEBUG 1 --- [   scheduling-1] c.g.s.j.cleanup.TagBasedCleanupSchedule  : Deleting TV shows and movies with tag: TagDeleteSchedule(tag=“90d”, expiration=PT2160H)

I noticed that it says tag=14d instead of tag="14d" like the rest of them. So I changed the tag of the series I'm testing with on Sonarr to 1d instead. Removed the container, spun it up again, and I still get similar logs.

03/22/2024
02:28:12 AM
2024-03-22T09:28:12.090Z  INFO 1 --- [           main] c.g.s.janitorr.JanitorrApplicationKt     : Starting JanitorrApplicationKt using Java 21.0.2 with PID 1 (/app/classes started by root in /)
03/22/2024
02:28:12 AM
2024-03-22T09:28:12.094Z DEBUG 1 --- [           main] c.g.s.janitorr.JanitorrApplicationKt     : Running with Spring Boot v3.2.2, Spring v6.1.3
03/22/2024
02:28:12 AM
2024-03-22T09:28:12.095Z  INFO 1 --- [           main] c.g.s.janitorr.JanitorrApplicationKt     : No active profile set, falling back to 1 default profile: "default"
03/22/2024
02:28:17 AM
2024-03-22T09:28:17.733Z  INFO 1 --- [           main] o.s.b.w.embedded.tomcat.TomcatWebServer  : Tomcat initialized with port 8978 (http)
03/22/2024
02:28:17 AM
2024-03-22T09:28:17.782Z  INFO 1 --- [           main] o.apache.catalina.core.StandardService   : Starting service [Tomcat]
03/22/2024
02:28:17 AM
2024-03-22T09:28:17.782Z  INFO 1 --- [           main] o.apache.catalina.core.StandardEngine    : Starting Servlet engine: [Apache Tomcat/10.1.18]
03/22/2024
02:28:18 AM
2024-03-22T09:28:18.110Z  INFO 1 --- [           main] o.a.c.c.C.[Tomcat].[localhost].[/]       : Initializing Spring embedded WebApplicationContext
03/22/2024
02:28:18 AM
2024-03-22T09:28:18.111Z  INFO 1 --- [           main] w.s.c.ServletWebServerApplicationContext : Root WebApplicationContext: initialization completed in 5802 ms
03/22/2024
02:28:18 AM
Standard Commons Logging discovery in action with spring-jcl: please remove commons-logging.jar from classpath in order to avoid potential conflicts
03/22/2024
02:28:21 AM
2024-03-22T09:28:21.739Z  INFO 1 --- [           main] o.s.b.w.embedded.tomcat.TomcatWebServer  : Tomcat started on port 8978 (http) with context path ''
03/22/2024
02:28:21 AM
2024-03-22T09:28:21.787Z  INFO 1 --- [           main] c.g.s.janitorr.JanitorrApplicationKt     : Started JanitorrApplicationKt in 12.141 seconds (process running for 17.147)
03/22/2024
02:28:22 AM
2024-03-22T09:28:22.020Z DEBUG 1 --- [   scheduling-1] c.g.s.j.cleanup.TagBasedCleanupSchedule  : Deleting TV shows and movies with tag: TagDeleteSchedule(tag=“1d”, expiration=PT24H)
03/22/2024
02:29:49 AM
2024-03-22T09:29:49.662Z DEBUG 1 --- [   scheduling-1] c.g.s.j.cleanup.TagBasedCleanupSchedule  : Deleting TV shows and movies with tag: TagDeleteSchedule(tag=14d, expiration=PT336H)
03/22/2024
02:30:54 AM
2024-03-22T09:30:54.173Z DEBUG 1 --- [   scheduling-1] c.g.s.j.cleanup.TagBasedCleanupSchedule  : Deleting TV shows and movies with tag: TagDeleteSchedule(tag=“30d”, expiration=PT720H)
03/22/2024
02:31:33 AM
2024-03-22T09:31:33.792Z DEBUG 1 --- [   scheduling-1] c.g.s.j.cleanup.TagBasedCleanupSchedule  : Deleting TV shows and movies with tag: TagDeleteSchedule(tag=“60d”, expiration=PT1440H)
03/22/2024
02:31:44 AM
2024-03-22T09:31:44.784Z DEBUG 1 --- [   scheduling-1] c.g.s.j.cleanup.TagBasedCleanupSchedule  : Deleting TV shows and movies with tag: TagDeleteSchedule(tag=“90d”, expiration=PT2160H)

I then removed all quotation marks from the tags in the application.yml file. I still get similar logs.

03/22/2024
02:33:31 AM
2024-03-22T09:33:31.309Z  INFO 1 --- [           main] c.g.s.janitorr.JanitorrApplicationKt     : Starting JanitorrApplicationKt using Java 21.0.2 with PID 1 (/app/classes started by root in /)
03/22/2024
02:33:31 AM
2024-03-22T09:33:31.321Z DEBUG 1 --- [           main] c.g.s.janitorr.JanitorrApplicationKt     : Running with Spring Boot v3.2.2, Spring v6.1.3
03/22/2024
02:33:31 AM
2024-03-22T09:33:31.322Z  INFO 1 --- [           main] c.g.s.janitorr.JanitorrApplicationKt     : No active profile set, falling back to 1 default profile: "default"
03/22/2024
02:33:42 AM
2024-03-22T09:33:42.351Z  INFO 1 --- [           main] o.s.b.w.embedded.tomcat.TomcatWebServer  : Tomcat initialized with port 8978 (http)
03/22/2024
02:33:42 AM
2024-03-22T09:33:42.551Z  INFO 1 --- [           main] o.apache.catalina.core.StandardService   : Starting service [Tomcat]
03/22/2024
02:33:42 AM
2024-03-22T09:33:42.552Z  INFO 1 --- [           main] o.apache.catalina.core.StandardEngine    : Starting Servlet engine: [Apache Tomcat/10.1.18]
03/22/2024
02:33:44 AM
2024-03-22T09:33:44.412Z  INFO 1 --- [           main] o.a.c.c.C.[Tomcat].[localhost].[/]       : Initializing Spring embedded WebApplicationContext
03/22/2024
02:33:44 AM
2024-03-22T09:33:44.439Z  INFO 1 --- [           main] w.s.c.ServletWebServerApplicationContext : Root WebApplicationContext: initialization completed in 12712 ms
03/22/2024
02:33:44 AM
Standard Commons Logging discovery in action with spring-jcl: please remove commons-logging.jar from classpath in order to avoid potential conflicts
03/22/2024
02:34:00 AM
2024-03-22T09:34:00.350Z  INFO 1 --- [           main] o.s.b.w.embedded.tomcat.TomcatWebServer  : Tomcat started on port 8978 (http) with context path ''
03/22/2024
02:34:00 AM
2024-03-22T09:34:00.766Z  INFO 1 --- [           main] c.g.s.janitorr.JanitorrApplicationKt     : Started JanitorrApplicationKt in 34.623 seconds (process running for 40.061)
03/22/2024
02:34:01 AM
2024-03-22T09:34:01.735Z DEBUG 1 --- [   scheduling-1] c.g.s.j.cleanup.TagBasedCleanupSchedule  : Deleting TV shows and movies with tag: TagDeleteSchedule(tag=1d, expiration=PT24H)
03/22/2024
02:36:09 AM
2024-03-22T09:36:09.266Z DEBUG 1 --- [   scheduling-1] c.g.s.j.cleanup.TagBasedCleanupSchedule  : Deleting TV shows and movies with tag: TagDeleteSchedule(tag=14d, expiration=PT336H)
03/22/2024
02:36:19 AM
2024-03-22T09:36:19.970Z DEBUG 1 --- [   scheduling-1] c.g.s.j.cleanup.TagBasedCleanupSchedule  : Deleting TV shows and movies with tag: TagDeleteSchedule(tag=30d, expiration=PT720H)
03/22/2024
02:36:38 AM
2024-03-22T09:36:38.776Z DEBUG 1 --- [   scheduling-1] c.g.s.j.cleanup.TagBasedCleanupSchedule  : Deleting TV shows and movies with tag: TagDeleteSchedule(tag=60d, expiration=PT1440H)
03/22/2024
02:36:47 AM
2024-03-22T09:36:47.396Z DEBUG 1 --- [   scheduling-1] c.g.s.j.cleanup.TagBasedCleanupSchedule  : Deleting TV shows and movies with tag: TagDeleteSchedule(tag=90d, expiration=PT2160H)

Am I maybe not tagging the series in Sonarr correctly? I'm just going to the series in Sonarr, hitting "Edit," adding the tag 1d, hitting Enter on the keyboard, seeing it pop up as a blue tag that says 1d, then hitting "Save."

I've also double-checked and yes, janitorr sees my TV shows in the exact same way as Sonarr:

- "/Volumes/ExternalHardDrive/TV Shows:/tv-shows"
Schaka commented 7 months ago

Can you screenshot your history of that show in Sonarr? Everything still looks correct to me.

It doesn't really do anything magical. It matches tags by name and marks shows for deletion if they are old enough and match the tag and its expiration time.

nothing2obvi commented 7 months ago

So far I've tested it on 2 shows. Testing with the 14d tag, with an expiration of 14d.

Does Janitorr delete individual episodes or just seasons / entire series? Maybe that's the problem for this first one.

This first show was grabbed and sent to the remote download client by Sonarr, then downloaded from the remote client by FTPGrab, and sent to the appropriate media folder by Hazel. Screen Shot 2024-03-22 at 10 45 37 AM

This second show was manually downloaded and manually added to the appropriate media folder. I'm positive it was added to Sonarr at least a year ago. Screen Shot 2024-03-22 at 10 52 08 AM

Schaka commented 7 months ago

It only goes season by season. As the latest grab for that first show is 2-3 days old, it won't delete.

nothing2obvi commented 7 months ago

Okay. Sorry to waste your time. I appreciate the help and clarification.

So for the 2nd series, it won't delete because there is no history, right? Since Sonarr didn't grab the downloads itself and I manually added the torrent to the client, let it download, then imported the files into Sonarr? Because that series is at least 1 year old.

Schaka commented 7 months ago

Correct, if not history is available, it won't delete anything.

You'll have to wait until I can find the time to implement JellyStat as a solution to update history timestamps (i.e. overwrite with latest watch).

For future reference, you can easily add torrent to qbit using the category set for that client in Sonarr and it'll automatically import everything correctly. It's not required to let Sonarr grab it automatically.

nothing2obvi commented 7 months ago

I see. Yeah I've been doing that with the categories/labels lately. Thanks. I'll be back as this develops!