Closed Andy2244 closed 7 years ago
Hi, and thank you for the PR :)
I like it, but before I merge it I think we should consider an alternative path for the animetitles.xml.gz-file. I don't know how windows behave, so I'll speak from the unix perspective: The problem with using tempfile.gettempdir() is that it will on almost always return /tmp on unix-systems. The /tmp directory is commonly cleaned on reboot and/or at regular intervals. /var/tmp which I used previously is not as heavily purged (although there are systems that does that too).
Anidb enforce (very) heavy rate limiting on that file; and fetching it too often may get you permanently banned, so it's quite important that it is not fetched too often. It would be quite miserable if users of the library got banned from anidb just because they rebooted their computer one time too many.
On unix, I think a better place for it would be in the user home directory like so: "~/.adbb/animetitles.xml.gz", could that work for windows as well?
We could just use os.getcwd()
and leave it in the cmd directory, flexget also stores its own log/sqlite.db this way. We could also check for windows specifically and use a separate code path?
I would also like to change the way the file is updated. Anidb allows a once per day update frequency. So in my other project i was checking vs 1 day and than compared the remote size with the local size and if it differs it got the new version. This ensures you always have a updated title file, without getting banned.
PS: Btw what you mean by 'permanently banned'? I use jmm/shoko for years and as far as i know Anidb never permanently banns you, those are always just 12-24 hour temp banns.
I think the users home/data dir works too, need to check what is safe to get under python.
oki, i think a os specific solution works best. I remembered that flexget might be running on openwrt, where you also want to use '/var/tmp', instead of some data, home location.
Thanks, I've merged it now. It would be good if you could keep the PRs a bit shorter in the future though. Preferably one issue per PR as it's easier to review them that way :)
I can't say I know exactly how anidb banning-policies are, but when writing this library I got multiple bans; most of them were short temp-bans against the API; but when I got banned from the animetitles.xml-file (actually I think I got banned for all anidb http traffic) I had to crawl to the IRC channel and ask for forgiveness before I was unbanned, and I'm pretty sure I waited over 24 hours before that...
You're right about the wiki saying once per day though, so perhaps one week is a bit much. Checking remote size and/or timestamp to decide if it should be redownloaded would also be a good thing. I would definitely merge a PR about that :)
yeah, i missed the checkbox that automatically reformats all to PEP8 in pyCharm :) I was just working on the remote check and will make a shorter pull request next time.
thx for the merge
Wow it seems my ip also just got banned, after a second download attempt. I cant reach the http site at all atm, but works over vpn :p
I guess i will see tomorrow if the ip ban is lifted, since i should have the same ip tomorrow.
Hi, and thanks for the nice library, so i don't have to write a new one for a new flexget anidb plugin.
I had to-do some small changes to make it run under windows/flexget, so just sending them here.
thx Andy