search

haijeploeg / excludarr

Excludarr manages your libraries in Radarr/Sonarr. It keeps track of your library and checks if your movies and series are also available on a configured streaming provider. It can exclude the movies and series that are available on a configured streaming provider. But it can also re-add movies and series if they are not streaming anymore.
MIT License
197 stars 12 forks source link

Add database support #43

Open haijeploeg opened 2 years ago

haijeploeg commented 2 years ago

Is your feature request related to a problem? Please describe. Right now Excludarr is pulling all the data from JustWatch and doing multiple searches at JustWatch to gather the justwatch ID. This is causing an overload of API calls at JustWatch, when you only need to pull this data once in a while and not everytime you run the script. Big libraries take a lot of time to process right now. Also the re-add function only works with not-monitored media. If we keep a local database in sync with Radarr and Sonarr, we can also use the re-add functionality on deleted media.

Describe the solution you'd like Sync all records of Sonarr and Radarr to a database with the JustWatch ID and the streaming providers for the configured locale. Using this principle we can fetch all the information from the database and speed up the process. Syncing the database once or twice a week will be sufficient. Also add a flag to force a sync before exclude or re-add the movies/series.

Describe alternatives you've considered A web application will be too big for the project to maintain. Another solution is to store a file based database like json files over the system. But this will be too hard to maintain for big libraries and will possible result in high IO.

Additional context None

haijeploeg commented 2 years ago

Tried to do this with a database, but it is really hard to keep that database up to date. The database needs to be always in sync with Radarr/Sonarr. I tried a PoC, but in the end it was too complex, too hard to maintain and it tooks longer.

Another solution is to do this with labels on the Radarr/Sonarr entries. Once the Justwatch ID is found, place a label with the JustWatchID on the Radarr/Sonarr entry. Everytime a search is done, look for this label, otherwise find it and create the label. This will speed up the next few iterations as the JustWatchID is already found.

faulander commented 1 year ago

i will fork excludarr and implement a json file with just the sonarr id and the justwatch id. Let's see how much i/o this really is. Whenever excludarr is run, first it checks if all justwatch ids are still in sonarr, if not clean the json file. then get the missing justwatch id's. I am not sure what you mean by it's hard to keep them synced, you don't need to. It's enough and should speed up the process considerably when running excludarr.