search

sureshfizzy / CineSync

CineSync is a Python based library management tool designed to organize debrid & local libraries without the support of Sonarr & Radarr
77 stars 7 forks source link

[FEAUTURE] More Docker options #20

Closed snucker-fod closed 2 months ago

snucker-fod commented 2 months ago

Would it be possible to convert all of the various script options as docker variables for the next release?

sureshfizzy commented 2 months ago

Would it be possible to convert all of the various script options as docker variables for the next release?

can you be more specific ?

snucker-fod commented 2 months ago

image

Having variables to enable these features at set intervals would be cool.

FULL_LIBRARY_SCAN=true FULL_LIBRARY_SCAN_INTERVAL=1440 #in seconds DELETE_BROKEN_SYMLINKS=true DELETE_BROKEN_SYMLINKS_INTERVAL=1440 #in seconds

etc

snucker-fod commented 2 months ago

And maybe the ability to fine tune where symlinks end up? I think for me personally I don't need _as _much__ seperation between everything, just movies/shows is fine.

FullHD Retro480p SDClassics

sureshfizzy commented 2 months ago

I believe everything is managed via Docker, isn't it ?

1) The initial docker compose will trigger a full scan based on the database. If the database is not present, a full scan will occur. If the database is present, it will skip the full scan.

2) The functionality for deleting broken symlinks is already included, along with real-time monitoring in Docker. By default, the sleep time is set to 60 seconds, but you can increase it by adjusting the .env file.

Regarding the separation structure, I have implemented a new structure in the symlinks codebase that I believe addresses your issue. I haven’t pushed these changes to Docker yet, but you can build it locally and use it if needed.

snucker-fod commented 2 months ago

Awesome! For point #1, I think it could be nice to force a full scan on demand, not only when the container is created.

sureshfizzy commented 2 months ago

Awesome! For point #1, I think it could be nice to force a full scan on demand, not only when the container is created.

I don't see the necessary of it or I might be missing some point of views (correct me If I'm wrong)

snucker-fod commented 2 months ago

I guess it depends how the DB handles scans, if I delete a symlink that CineSync already processed, does it skip it the next time it runs?

sureshfizzy commented 2 months ago

I guess it depends how the DB handles scans, if I delete a symlink that CineSync already processed, does it skip it the next time it runs?

No it doesn't, it will check for the particular file if it is available or not, if yes then it will re-pick them. Additionally if there are new files on the next run it will also pick them. That's why I don't see the necessity of running a full scan again.

snucker-fod commented 2 months ago

Good to know, thanks!

snucker-fod commented 2 months ago

Going to close this then.

idanlah commented 1 month ago

@sureshfizzy but i don't think the docker version deletes empty folders. it does clean up broken symlinks but it doesn't delete the folder itself. i believe this logic is in CineSync.py and not in main.py i think it should also be added to docker

sureshfizzy commented 1 month ago

@sureshfizzy but i don't think the docker version deletes empty folders. it does clean up broken symlinks but it doesn't delete the folder itself. i believe this logic is in CineSync.py and not in main.py i think it should also be added to docker

I had a hunch on this but couldn't check lately since I was busy with other things, I'll update with a fresh patch soon.

idanlah commented 1 month ago

I believe everything is managed via Docker, isn't it ?

  1. The initial docker compose will trigger a full scan based on the database. If the database is not present, a full scan will occur. If the database is present, it will skip the full scan.
  2. The functionality for deleting broken symlinks is already included, along with real-time monitoring in Docker. By default, the sleep time is set to 60 seconds, but you can increase it by adjusting the .env file.

Regarding the separation structure, I have implemented a new structure in the symlinks codebase that I believe addresses your issue. I haven’t pushed these changes to Docker yet, but you can build it locally and use it if needed.

@sureshfizzy thanks, i think its worth thinking about a push design and not just polling evert x seconds. if i update my RD rclone i want that to be updated immediately in symlink and in my plex and not after 60 seconds. its pretty easy to do it in the script but don't know how to do it in docker.

also, an after_script can help with updating plex/jellyfin etc.. after a symlink has been updated

sureshfizzy commented 1 month ago

I believe everything is managed via Docker, isn't it ?

  1. The initial docker compose will trigger a full scan based on the database. If the database is not present, a full scan will occur. If the database is present, it will skip the full scan.
  2. The functionality for deleting broken symlinks is already included, along with real-time monitoring in Docker. By default, the sleep time is set to 60 seconds, but you can increase it by adjusting the .env file.

Regarding the separation structure, I have implemented a new structure in the symlinks codebase that I believe addresses your issue. I haven’t pushed these changes to Docker yet, but you can build it locally and use it if needed.

@sureshfizzy thanks, i think its worth thinking about a push design and not just polling evert x seconds. if i update my RD rclone i want that to be updated immediately in symlink and in my plex and not after 60 seconds. its pretty easy to do it in the script but don't know how to do it in docker.

also, an after_script can help with updating plex/jellyfin etc.. after a symlink has been updated

We can update on the env right ? it does reflect the polling activity.

https://github.com/sureshfizzy/CineSync/blob/main/.env#L78

Setting the timer here to 5 or 10 seconds works. I used polling here because inotify realtime monitor only works on local filesystems, and not on mounted directories. Due to this, instant reflection is not feasible, although polling still works well.

sureshfizzy commented 1 month ago

@idanlah Grab the latest image, empty folders and broken symlinks will be erased. Here is the tested output.

2024-10-24 09:41:09 [DEBUG] Deleting broken symlink: /home/suresh/projects/CineSync/Movies/FullHD/100% Wolf (2020)/100% Wolf (2020) (1080p BluRay x265 HEVC 10bit AAC 5.1 Tigole).mkv

2024-10-24 09:41:09 [DEBUG] File found in database: /home/suresh/projects/CineSync/Movies/FullHD/100% Wolf (2020)/100% Wolf (2020) (1080p BluRay x265 HEVC 10bit AAC 5.1 Tigole).mkv

2024-10-24 09:41:09 [DEBUG] Removing /home/suresh/projects/CineSync/Movies/FullHD/100% Wolf (2020)/100% Wolf (2020) (1080p BluRay x265 HEVC 10bit AAC 5.1 Tigole).mkv from database.

2024-10-24 09:41:09 [DEBUG] Deleting empty folder: /home/suresh/projects/CineSync/Movies/FullHD/100% Wolf (2020)

idanlah commented 1 month ago

@idanlah Grab the latest image, empty folders and broken symlinks will be erased. Here is the tested output.

2024-10-24 09:41:09 [DEBUG] Deleting broken symlink: /home/suresh/projects/CineSync/Movies/FullHD/100% Wolf (2020)/100% Wolf (2020) (1080p BluRay x265 HEVC 10bit AAC 5.1 Tigole).mkv

2024-10-24 09:41:09 [DEBUG] File found in database: /home/suresh/projects/CineSync/Movies/FullHD/100% Wolf (2020)/100% Wolf (2020) (1080p BluRay x265 HEVC 10bit AAC 5.1 Tigole).mkv

2024-10-24 09:41:09 [DEBUG] Removing /home/suresh/projects/CineSync/Movies/FullHD/100% Wolf (2020)/100% Wolf (2020) (1080p BluRay x265 HEVC 10bit AAC 5.1 Tigole).mkv from database.

2024-10-24 09:41:09 [DEBUG] Deleting empty folder: /home/suresh/projects/CineSync/Movies/FullHD/100% Wolf (2020)

@sureshfizzy Thanks, it works great for movies but for shows it doesn't delete the parent show folder (only the season folders)

sureshfizzy commented 4 weeks ago

@idanlah Grab the latest image, empty folders and broken symlinks will be erased. Here is the tested output. 2024-10-24 09:41:09 [DEBUG] Deleting broken symlink: /home/suresh/projects/CineSync/Movies/FullHD/100% Wolf (2020)/100% Wolf (2020) (1080p BluRay x265 HEVC 10bit AAC 5.1 Tigole).mkv 2024-10-24 09:41:09 [DEBUG] File found in database: /home/suresh/projects/CineSync/Movies/FullHD/100% Wolf (2020)/100% Wolf (2020) (1080p BluRay x265 HEVC 10bit AAC 5.1 Tigole).mkv 2024-10-24 09:41:09 [DEBUG] Removing /home/suresh/projects/CineSync/Movies/FullHD/100% Wolf (2020)/100% Wolf (2020) (1080p BluRay x265 HEVC 10bit AAC 5.1 Tigole).mkv from database. 2024-10-24 09:41:09 [DEBUG] Deleting empty folder: /home/suresh/projects/CineSync/Movies/FullHD/100% Wolf (2020)

@sureshfizzy Thanks, it works great for movies but for shows it doesn't delete the parent show folder (only the season folders)

Oh my bad, I need to enhance a bit, I'll do it and let you know.

sureshfizzy commented 2 weeks ago

@idanlah these have been updated, everything will be working well, you can pull the new docker images.

idanlah commented 1 week ago

@sureshfizzy thank you very much. is it possible to maybe add a parameter in .env to disable or enable deletion of symlinks? im asking because sometimes the rclone container is stopped or after restart it didn't start yet and cinesync keeps deleting all symlinks because it can't connect yet to rclone

sureshfizzy commented 1 week ago

@sureshfizzy thank you very much. is it possible to maybe add a parameter in .env to disable or enable deletion of symlinks? im asking because sometimes the rclone container is stopped or after restart it didn't start yet and cinesync keeps deleting all symlinks because it can't connect yet to rclone

can you create a new issue and add your observations there

idanlah commented 1 week ago

@sureshfizzy sure, added #31 Thanks