search

pymedusa / Medusa

Automatic Video Library Manager for TV Shows. It watches for new episodes of your favorite shows, and when they are posted it does its magic.
https://pymedusa.com
GNU General Public License v3.0
1.8k stars 276 forks source link

AniDB cacheing into local DB #8221

Closed crypticpiper closed 4 years ago

crypticpiper commented 4 years ago

Is your feature request related to a problem? Please describe. It seems when adding or searching a new anime, it seems that medusa calls to the api every time and then your AniDB account gets a temp ban placed on it because of excessive api use.

Describe the solution you'd like I would suggest to have an option for medusa to locally cache Anidb's DB and update the cache on a time based interval to avoid account bans.

Describe alternatives you've considered An alternative to the above would be to have an option to throttle how many times medusa calls out to the api and once reaching that limit then defaulting back to the other indexers(TVDB,etc) but this kinda just seems like a bandaid to the problem.

p0psicles commented 4 years ago

We use dogpile caching as much as possible. But maybe we missed something. There is an expiration to the cache though

p0psicles commented 4 years ago

Btw, it shouldn't connect to anidb on searching.

crypticpiper commented 4 years ago

Well what is the expiration on the cache? Also just recently going through the logs and on searchqueue-backlog, afer hitting the indexers that were added it also then Performed an episode search for that specific anime from AniDB.

p0psicles commented 4 years ago

Do you have a Log for that?

crypticpiper commented 4 years ago

Yeah I copied the section out of the logs when I seen it.

2020-06-29 02:23:43 INFO SEARCHQUEUE-BACKLOG-293267 :: [NzbHydra2] :: [] Episodes [2] of season 1 are needed with this quality for Prison School 2020-06-29 02:23:43 INFO SEARCHQUEUE-BACKLOG-293267 :: [NzbHydra2] :: [] Picked Prison.School.S01E02.720p.WEBRip.AAC2.0.H.264-BTN as the best result. 2020-06-29 02:23:43 INFO SEARCHQUEUE-BACKLOG-293267 :: [NzbHydra2] :: [] Quality of Prison.School.S01E03.1080p.Blu-Ray.10-Bit.Dual-Audio.TrueHD.x265-iAHD is 1080p BluRay 2020-06-29 02:23:43 INFO SEARCHQUEUE-BACKLOG-293267 :: [NzbHydra2] :: [] Quality of Prison.School.S01E03.1080p.Blu-Ray.10-Bit.Dual-Audio.TrueHD.x265-iAHD is 1080p BluRay 2020-06-29 02:23:43 INFO SEARCHQUEUE-BACKLOG-293267 :: [NzbHydra2] :: [] Quality of Prison.School.S01E04.1080p.Blu-Ray.10-Bit.Dual-Audio.TrueHD.x265-iAHD is 1080p BluRay 2020-06-29 02:23:43 INFO SEARCHQUEUE-BACKLOG-293267 :: [NzbHydra2] :: [] Quality of Prison.School.S01E04.1080p.Blu-Ray.10-Bit.Dual-Audio.TrueHD.x265-iAHD is 1080p BluRay 2020-06-29 02:23:43 INFO SEARCHQUEUE-BACKLOG-293267 :: [NzbHydra2] :: [] Quality of Prison.School.S01E05.1080p.Blu-Ray.10-Bit.Dual-Audio.TrueHD.x265-iAHD is 1080p BluRay 2020-06-29 02:23:43 INFO SEARCHQUEUE-BACKLOG-293267 :: [NzbHydra2] :: [] Quality of Prison.School.S01E05.1080p.Blu-Ray.10-Bit.Dual-Audio.TrueHD.x265-iAHD is 1080p BluRay 2020-06-29 02:23:43 INFO SEARCHQUEUE-BACKLOG-293267 :: [NzbHydra2] :: [] Quality of Prison.School.S01E06.1080p.Blu-Ray.10-Bit.Dual-Audio.TrueHD.x265-iAHD is 1080p BluRay 2020-06-29 02:23:43 INFO SEARCHQUEUE-BACKLOG-293267 :: [NzbHydra2] :: [] Quality of Prison.School.S01E06.1080p.Blu-Ray.10-Bit.Dual-Audio.TrueHD.x265-iAHD is 1080p BluRay 2020-06-29 02:23:43 INFO SEARCHQUEUE-BACKLOG-293267 :: [NzbHydra2] :: [] Quality of Prison.School.S01E07.1080p.Blu-Ray.10-Bit.Dual-Audio.TrueHD.x265-iAHD is 1080p BluRay 2020-06-29 02:23:43 INFO SEARCHQUEUE-BACKLOG-293267 :: [NzbHydra2] :: [] Quality of Prison.School.S01E07.1080p.Blu-Ray.10-Bit.Dual-Audio.TrueHD.x265-iAHD is 1080p BluRay 2020-06-29 02:23:43 INFO SEARCHQUEUE-BACKLOG-293267 :: [NzbHydra2] :: [] Quality of Prison.School.S01E08.1080p.Blu-Ray.10-Bit.Dual-Audio.TrueHD.x265-iAHD is 1080p BluRay 2020-06-29 02:23:43 INFO SEARCHQUEUE-BACKLOG-293267 :: [NzbHydra2] :: [] Quality of Prison.School.S01E08.1080p.Blu-Ray.10-Bit.Dual-Audio.TrueHD.x265-iAHD is 1080p BluRay 2020-06-29 02:23:43 INFO SEARCHQUEUE-BACKLOG-293267 :: [NzbHydra2] :: [] Quality of Prison.School.S01E09.1080p.Blu-Ray.10-Bit.Dual-Audio.TrueHD.x265-iAHD is 1080p BluRay 2020-06-29 02:23:43 INFO SEARCHQUEUE-BACKLOG-293267 :: [NzbHydra2] :: [] Quality of Prison.School.S01E09.1080p.Blu-Ray.10-Bit.Dual-Audio.TrueHD.x265-iAHD is 1080p BluRay 2020-06-29 02:23:43 INFO SEARCHQUEUE-BACKLOG-293267 :: [NzbHydra2] :: [] Quality of Prison.School.S01E10.1080p.Blu-Ray.10-Bit.Dual-Audio.TrueHD.x265-iAHD is 1080p BluRay 2020-06-29 02:23:43 INFO SEARCHQUEUE-BACKLOG-293267 :: [NzbHydra2] :: [] Quality of Prison.School.S01E10.1080p.Blu-Ray.10-Bit.Dual-Audio.TrueHD.x265-iAHD is 1080p BluRay 2020-06-29 02:23:43 INFO SEARCHQUEUE-BACKLOG-293267 :: [NzbHydra2] :: [] Quality of Prison.School.S01E11.1080p.Blu-Ray.10-Bit.Dual-Audio.TrueHD.x265-iAHD is 1080p BluRay 2020-06-29 02:23:43 INFO SEARCHQUEUE-BACKLOG-293267 :: [NzbHydra2] :: [] Quality of Prison.School.S01E11.1080p.Blu-Ray.10-Bit.Dual-Audio.TrueHD.x265-iAHD is 1080p BluRay 2020-06-29 02:23:43 INFO SEARCHQUEUE-BACKLOG-293267 :: [NzbHydra2] :: [] Quality of Prison.School.S01E12.1080p.Blu-Ray.10-Bit.Dual-Audio.TrueHD.x265-iAHD is 1080p BluRay 2020-06-29 02:23:43 INFO SEARCHQUEUE-BACKLOG-293267 :: [NzbHydra2] :: [] Quality of Prison.School.S01E12.1080p.Blu-Ray.10-Bit.Dual-Audio.TrueHD.x265-iAHD is 1080p BluRay 2020-06-29 02:23:43 INFO SEARCHQUEUE-BACKLOG-293267 :: [Anizb] :: [] Performing episode search for Prison School

p0psicles commented 4 years ago

Sorry but I don't see it? Maybe your confused with Anizb which is a provider. A different URL then anidb. there not affiliated.

p0psicles commented 4 years ago

I've checked. And only when searching for a new anime using an indexer (tvdb). That's not searching providers for torrents/nzb! It will try to get release groups from anidb's (udp API). It will cache all results in a file on disk. The cache is kept 3 days. Each time you open the editShow page for an anime, it will try to get it's release groups. So if you attempt to open the editShow page after 3 days, it will again request the anidb udp API.

p0psicles commented 4 years ago

We also request anidb http api, when you open the recommended shows -> anidb page. Also there, we apply dogpile caching. That is kept 7 days.

I'm closing this issue, as I don't see any evidence we hit the anidb API too much. If you can point me to something, I'll gladly reconsider and address.