winterbird-code / adbb

Object Oriented UDP Client Library for AniDB
GNU General Public License v3.0
17 stars 4 forks source link

[fix] make it flexget compatible #1

Closed Andy2244 closed 7 years ago

Andy2244 commented 7 years ago

Hi, and thanks for the nice library, so i don't have to write a new one for a new flexget anidb plugin.

I had to-do some small changes to make it run under windows/flexget, so just sending them here.

thx Andy

winterbird-code commented 7 years ago

Hi, and thank you for the PR :)

I like it, but before I merge it I think we should consider an alternative path for the animetitles.xml.gz-file. I don't know how windows behave, so I'll speak from the unix perspective: The problem with using tempfile.gettempdir() is that it will on almost always return /tmp on unix-systems. The /tmp directory is commonly cleaned on reboot and/or at regular intervals. /var/tmp which I used previously is not as heavily purged (although there are systems that does that too).

Anidb enforce (very) heavy rate limiting on that file; and fetching it too often may get you permanently banned, so it's quite important that it is not fetched too often. It would be quite miserable if users of the library got banned from anidb just because they rebooted their computer one time too many.

On unix, I think a better place for it would be in the user home directory like so: "~/.adbb/animetitles.xml.gz", could that work for windows as well?

Andy2244 commented 7 years ago

We could just use os.getcwd() and leave it in the cmd directory, flexget also stores its own log/sqlite.db this way. We could also check for windows specifically and use a separate code path?

I would also like to change the way the file is updated. Anidb allows a once per day update frequency. So in my other project i was checking vs 1 day and than compared the remote size with the local size and if it differs it got the new version. This ensures you always have a updated title file, without getting banned.

PS: Btw what you mean by 'permanently banned'? I use jmm/shoko for years and as far as i know Anidb never permanently banns you, those are always just 12-24 hour temp banns.

Andy2244 commented 7 years ago

I think the users home/data dir works too, need to check what is safe to get under python.

Andy2244 commented 7 years ago

oki, i think a os specific solution works best. I remembered that flexget might be running on openwrt, where you also want to use '/var/tmp', instead of some data, home location.

winterbird-code commented 7 years ago

Thanks, I've merged it now. It would be good if you could keep the PRs a bit shorter in the future though. Preferably one issue per PR as it's easier to review them that way :)

I can't say I know exactly how anidb banning-policies are, but when writing this library I got multiple bans; most of them were short temp-bans against the API; but when I got banned from the animetitles.xml-file (actually I think I got banned for all anidb http traffic) I had to crawl to the IRC channel and ask for forgiveness before I was unbanned, and I'm pretty sure I waited over 24 hours before that...

You're right about the wiki saying once per day though, so perhaps one week is a bit much. Checking remote size and/or timestamp to decide if it should be redownloaded would also be a good thing. I would definitely merge a PR about that :)

Andy2244 commented 7 years ago

yeah, i missed the checkbox that automatically reformats all to PEP8 in pyCharm :) I was just working on the remote check and will make a shorter pull request next time.

thx for the merge

Andy2244 commented 7 years ago

Wow it seems my ip also just got banned, after a second download attempt. I cant reach the http site at all atm, but works over vpn :p

I guess i will see tomorrow if the ip ban is lifted, since i should have the same ip tomorrow.