search

pandabuilder / pandachaika

GNU General Public License v3.0
112 stars 9 forks source link

Exceed eh limit easily when using eh matching. #17

Closed northtowertop closed 2 years ago

northtowertop commented 2 years ago

As the title, just after matching one or two doujinshis using eh matcher(title or cover), the logger will show as follows, and then all the matches after will have no results.

[core.providers.panda.parsers] Could not parse response to JSON: Your IP address has been temporarily banned for excessive pageloads which indicates that you are using automated mirroring/harvesting software. The ban expires in 23 hours and 59 minutes

I am using the default ini settings and have set the eh/ex cookies(but I guess pandachaika doesn't use cookies when doing matching). And not using VPN. If the match worker only call for eh api and asking a meta result in json, it is wierd why this will happen...............

pandabuilder commented 2 years ago

The EH API doesn't support any type of search, it's all done parsing the main page with search terms. The API is used to fetch the metadata associated with the possible matches.

I haven't really run into ban issues, even with the 6 seconds default delay for API calls. But that might vary per account or while being anonymous. So you might need to increase that value, for eh sets the wait period between API calls. The search delay when comparing is a separate value (time_to_wait_after_compare), and that's 10 seconds for eh.

Anyways, sorry if you have run into bans, but as I said, I haven't triggered one in many instances of this setup for a long time.

northtowertop commented 2 years ago

Thanks for the reply and the introduction. I think I should take some time to figure out what's happend, cause me and my friends all have this problem. BTW, really nice project for managing my messy stuff XD.

northtowertop commented 2 years ago

And I noticed that even if the pandachaika shows an ip ban in log, I could still using eh with an eh account. But after I logout my account, an ip ban will appear. Maybe I didn't set the cookies correctly and the pandachaika couldn't do the matching job with a valid eh account.

northtowertop commented 2 years ago

Done. It is the cookies problem, I forgot to import the igneous and sk.

northtowertop commented 2 years ago

But my Gold Star account still got banned after three matches with double delay time compared with default setting......... Anyway, at least I know the reason. Thx.