Schaka / janitorr

Cleans your Radarr, Sonarr, Jellyseerr and Jellyfin before you run out of space
GNU General Public License v3.0
263 stars 5 forks source link

Symlinks not created #10

Closed sjafferali closed 6 months ago

sjafferali commented 6 months ago

After setting this up and running it, the leaving soon collections are created in the filesystem and in jellyfin, however symlinks for the files that are leaving soon are not getting created, leading to both collections always being empty in jellyfin. Attempted running this with dry-run = False as well as dry-run = True. Filesystem access is also set to true in my configuration.

Running docker image v0.9.3.

Schaka commented 6 months ago

Can you try v0.9.3b, please? That should be the image tagged latest. I'm not sure why GitHub doesn't order them correctly. I'll probably stop using letters.

There was an intermediate bug where always re-creating links (the config setting) would prevent it from being created.

If v0.9.3b still causes issues, are you sure that:

sjafferali commented 6 months ago

Hey @Schaka . I tried v0.9.3b, but symlinks are still not being created.

If I increase the movie-expiration to something that matches some movies (running with dry-run = True), I do see that it logs deletion messages, so not sure why the symlink creation is not also occurring since the leaving-soon setting should also be matching some of my content.

I am running it as root, and Jellyfin and Janitorr have the same volume mapping.

Schaka commented 6 months ago

I should probably adjust to docs to make it more clear how the options work. If you set movie expiration to 120 days and leaving-soon to 14 days, it will add all the movies that are 106 to 119 days old to the leaving soon collections, until they approach 120 when they get deleted.

For debugging purposes, you can try setting it it to something simple like:

Keep in mind that leaving-soon won't touch any any files more recent than 15-14 days (i.e. only one day old).

To add debug logging, you may add this at the top of your application.yml:

logging:
  level:
    com.github.schaka: TRACE

Currently, I only use debug messages, so DEBUG would be fine too.

sjafferali commented 6 months ago

Enabled trace logging and started running again with the below configuration:

logging:
  level:
    com.github.schaka: TRACE

server:
  port: 8978

# File system access (same mapping as Sonarr, Radarr and Jellyfin) is required to delete seasons by season and create "Leaving Soon" collections in Jellyfin
# Currently, Jellyfin does not support an easy way to add only a few seasons or movies to a collection, we need access to temporary hardlinks
# Additionally, checks to prevent deletion on currently still seeding media currently require file system access as well
file-system:
  access: true
  leaving-soon-dir: "/downloads/leaving-soon" # A directory this container can write to and Jellyfin can find under the same path - this will contain new folders with SymLinks to files for Jellyfin's "Leaving Soon" collections
  from-scratch: true # Clean up entire directory and rebuild from scratch - this can help with clearing orphaned data - turning this off can save resources (less writes to drive)

application:
  dry-run: true
  leaving-soon: 14d # 14 days before a movie is deleted, it gets added to a "Leaving Soon" type collection
  movie-expiration: 15d # 3 months - if a movie's files on your system are older than this, they will be deleted
  season-expiration: 15d # 2 months - if a season's files on your system are older than this, they will be deleted

clients:
  sonarr:
    url: "REDACTED"
    api-key: "REDACTED"
  radarr:
    url: "REDACTED"
    api-key: "REDACTED"
  jellyfin:
    enabled: true
    url: "REDACTED"
    api-key: "REDACTED"
    username: REDACTED
    password: REDACTED
  jellyseerr:
    enabled: true
    url: "REDACTED"
    api-key: "REDACTED"

# Currently not in use - intended to check seeding status and not delete currently seeding media
  torrent:
    type: QBITTORRENT
    name: qBittorrent
    auto-resume: true
    url: "locahost:8080"
    username: admin
    password: adminadmin

Log file that resulted: janitorr_redacted.txt

Schaka commented 6 months ago

I'm afraid this looks entirely correct to me.

Are you sure there's media on your server more recent than 14 days old? If you are, I think I have an idea what the issue could be. If you aren't, you could potentially set deletion time to 365d and leaving-soon to 364.

The code tries to find the folder as defined by Sonarr/Radarr. If that folder cannot be found (i.e. file access is not granted or cannot be found due to incorrect mapping), it will not create a link to it. Unfortunately, to create a link between the two files it needs access to the one the *arrs see.

So the assumption is also, that they contain the same mapping as Janitorr and Jellyfin. This is potentially something that could be changed by first finding the media via Jellyfin and then linking to the original path found in Jellyfin. I'm strongly considering implementing this as a fallback option.

However, if you need a fix sooner rather than later, you're probably better off getting the mappings correct.

Schaka commented 6 months ago

I did some more digging and it seems movies were not being created, whereas TV shows work(ed).

I don't know when exactly I broke it. Movie had previously worked for sure. I think it's time for some unit tests. I was hoping to avoid those for a private, hobbyist project, but it is what it is.

Please try v0.9.5, which has been working on my server.

sjafferali commented 6 months ago

Please try v0.9.5, which has been working on my server.

Yes, this one works! Although, while looking through the logs, I noticed multiple matches being returned from Jellyseer for each content item since IDs are re-used between sonar and radar . For example:

2024-02-27T15:29:02.130239629Z 2024-02-27T07:29:02.130-08:00  INFO 1 --- [   scheduling-1] c.g.s.j.j.JellyseerrRestService          : Found request: RequestResponse(createdAt=2024-02-17T23:58:23.000Z, id=52, is4k=false, media=Media(createdAt=2024-02-17T23:58:23.000Z, id=52, requests=null, status=4, imdbId=null, tmdbId=52814, tvdbId=366524, externalServiceId=4, jellyfinMediaId=null, updatedAt=2024-02-18T12:30:00.000Z), modifiedBy=ModifiedBy(avatar=/os_logo_square.png, createdAt=2024-01-13T20:35:16.000Z, email=REDACTED, id=1, permissions=2, requestCount=60, updatedAt=2024-01-13T20:35:16.000Z, userType=3, displayName=REDACTED, username=null, plexUsername=null, jellyfinUsername=REDACTED), profileId=1, requestedBy=ModifiedBy(avatar=/os_logo_square.png, createdAt=2024-01-13T20:35:16.000Z, email=REDACTED, id=1, permissions=2, requestCount=60, updatedAt=2024-01-13T20:35:16.000Z, userType=3, displayName=REDACTED, username=null, plexUsername=null, jellyfinUsername=REDACTED), rootFolder=/downloads/tvshows, serverId=0, status=2, updatedAt=2024-02-17T23:58:23.000Z, seasons=[RequestSeason(createdAt=2024-02-17T23:58:23.000Z, id=4, seasonNumber=1, status=2, updatedAt=2024-02-17T23:58:23.000Z), RequestSeason(createdAt=2024-02-17T23:58:23.000Z, id=5, seasonNumber=2, status=2, updatedAt=2024-02-17T23:58:23.000Z)], type=tv) for /downloads/movies/Napoleon (2023)/Napoleon.2023.2160p.WEB-DL.DDP5.1.Atmos.DV.HDR.H.265-FLUX[TGx].mkv | IMDB: tt13287846
2024-02-27T15:29:02.130304454Z 2024-02-27T07:29:02.130-08:00  INFO 1 --- [   scheduling-1] c.g.s.j.j.JellyseerrRestService          : Found request: RequestResponse(createdAt=2024-01-14T21:58:05.000Z, id=3, is4k=false, media=Media(createdAt=2024-01-14T21:58:05.000Z, id=3, requests=null, status=5, imdbId=null, tmdbId=753342, tvdbId=null, externalServiceId=4, jellyfinMediaId=4003b8536864c6a9ff14c4121623d844, updatedAt=2024-02-25T11:00:04.000Z), modifiedBy=ModifiedBy(avatar=/os_logo_square.png, createdAt=2024-01-13T20:35:16.000Z, email=REDACTED, id=1, permissions=2, requestCount=60, updatedAt=2024-01-13T20:35:16.000Z, userType=3, displayName=REDACTED, username=null, plexUsername=null, jellyfinUsername=REDACTED), profileId=7, requestedBy=ModifiedBy(avatar=/os_logo_square.png, createdAt=2024-01-13T20:35:16.000Z, email=REDACTED, id=1, permissions=2, requestCount=60, updatedAt=2024-01-13T20:35:16.000Z, userType=3, displayName=REDACTED, username=null, plexUsername=null, jellyfinUsername=REDACTED), rootFolder=/downloads/movies, serverId=0, status=2, updatedAt=2024-01-14T21:58:05.000Z, seasons=[], type=movie) for /downloads/movies/Napoleon (2023)/Napoleon.2023.2160p.WEB-DL.DDP5.1.Atmos.DV.HDR.H.265-FLUX[TGx].mkv | IMDB: tt13287846

Submitted https://github.com/Schaka/janitorr/pull/11 which fixes this matching behavior.

Schaka commented 6 months ago

Thanks for your contribution. Those changes and more are published in v0.9.6. I also publish every commit to develop into a new image now for simplified testing.

I'll keep the issue open for a bit to see if there's any more issues, but I think we can consider it fixed.