search

wwyaiykycnf / e621dl

The automated e621.net downloader
40 stars 23 forks source link

ValueError: No JSON object could be decoded - With Fix #46

Closed Inrixia closed 6 years ago

Inrixia commented 6 years ago

The Error: "ValueError: No JSON object could be decoded"

Whats actually happening: The request to e621 to get the posts in json format is failing. For me this error was due to e621 rejecting the default header set in the script to use "Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.0.12) Gecko/20070731 Ubuntu/dapper-security Firefox/1.5.0.12" As the user-agent making the request and giving a 403 Forbidden error.

To fix this I just changed the header defined in (\lib\support.py line 15) to "Mozilla/5.0 (Macintosh; Intel Mac OS X 10 7 4) AppleWebKit/537.13 (KHTML, like Gecko) Chrome/24.0.1290.1 Safari/537.13" Which e621 accepts.

This completely fixed the problem for me. I don't know if other issues were to do with the too large a request like noted in your documentation but perhaps this can help.

Inrixia commented 6 years ago

Just realized this is a duplicate of https://github.com/wwyaiykycnf/e621dl/issues/39.

Still this fix does work.

DatDraggy commented 6 years ago

Faking a useragent is not allowed by e621 and will get you banned again after some time.

We already know what the problem is. We're being rate limited on the api because only one request per second is allowed. I think I will just put a literal 1 sec delay in the api "crawler" which will pretty much slow everything down but at least won't get it banned again.