Comic Vine Scraper 1.0.89 does NOT work anymore!

[crullx]

Since yesterday, for about an entire day the Scraper hasn't been working. I use the latest ComicRack program, also using the latest Scraper. Also tried to reinstall the main program and the plugin too (none of them helped). Symptom: "Cannot Access Online Database". Before yesterday I was using it for days without any problems, so I know how to use it. But it stopped working yesterday. I also tried to press the "Restore Defaults" in the scraper. Didn't help. Does the developer know about this issue? Is a new/fixed version of the scraper coming up soon? Anybody's working on this issue right now? Oh and by the way I have two APIs. Tried both of them. Both give me the same error message.

[cbanack]

The Comic Vine Scraper currently works just fine for me. I would normally suggest that the Comic Vine API server just went down for a few hours, as sometimes happens, but I know that's not it because whenever that happens I get dozens of emails and message board postings in the ComicRack forums, and so far no one else is complaining.

Generally when someone sends me an angry and/or entitled sounding email complaining that the scraper has suddenly broken and that I should get to work fixing it immediately, the actual problem is one of two things:

1) the person has a firewall or some other kind of network issue that is blocking them from accessing the Comic Vine API (i.e. http://www.comicvine.com/api/)

2) the person has been abusing the Comic Vine API (for example trying to get around the API usage limits by using multiple API keys) and have had their IP address banned by Comic Vine.

Either problem will cause "Cannot Access Online Database" errors for you when no one else is having them. Can you rule out these two potential possibilities?

If not, try scraping a comic, and when you get the error message, press Ctrl-Shift-L. This will save an application log. Post the contents of that log here, and I'll see if it sheds any light on what is going wrong.

[crullx]

hello,

Thanks for your reply. It still doesn't work for me. Here's the log. Please tell me what you see on it: Oh by the way if I abused the API (which I don't even know how, because I haven't find any terms of usage about the API ANYWHERE on Comicvine!), then why does the error message says "please try again later"? If they banned my IP "please try again later" doesn't make any sense does it?

---

CV Scraper Version: 1.0.89 Running As: ComicRack Plugin (CR version 0.9.176) Cache Directory: C:\Users\sdsds\AppData\Roaming\Comic Vine Scraper\localCache

Settings File: C:\Users\sdsds\AppData\Roaming\Comic Vine Scraper\settings.dat

---

[X] Series [X] Volume [X] Number
[X] Title [X] Published [X] Released
[X] Crossovers [X] Publisher [X] Imprint
[X] Writer [X] Penciller [X] Inker
[X] Colorist [X] Letterer [X] Cover Art
[X] Editor [X] Summary [X] Characters

[X] Teams [X] Locations [X] Webpage

[X] Overwrite Existing [ ] Ignore Blanks
[X] Convert Imprints [ ] Autochoose Series
[X] Download Thumbs [X] Preserve Thumbs
[X] Confirm Issues [X] Rescraping: Notes
[X] Fast Rescrape [ ] Rescraping: Tags

[X] Summary Dialog

Do not show the initial 'Welcome Dialog'.

======> scraping next comic book: 'Hollywood Zombie Apocalypse v2014 #001 (2014-11).cbz' rescraping details in book identified its issue as: 'Issue #1 (469553)' ERROR OCCURRED CONTACTING COMICVINE. RETRYING... Error rescraping details: Caught DatabaseConnectionError: Comic Vine database could not be reached url: http://comicvine.com/api/issue/4000-469553/?api_key=...&client=cvscraper&format=xml CAUSE: System.Net.WebException: The server committed a protocol violation. Section=ResponseHeader Detail=CR must be followed by LF at System.Net.HttpWebRequest.GetResponse() at Microsoft.Scripting.Interpreter.FuncCallInstruction2.Run(InterpretedFrame frame) at Microsoft.Scripting.Interpreter.Interpreter.Run(InterpretedFrame frame) at Microsoft.Scripting.Interpreter.LightLambda.Run3[T0,T1,T2,TRet](T0 arg0, T1 arg1, T2 arg2) at System.Dynamic.UpdateDelegates.UpdateAndExecute2[T0,T1,TRet](CallSite site, T0 arg0, T1 arg1) at Microsoft.Scripting.Interpreter.FuncCallInstruction5.Run(InterpretedFrame frame) at Microsoft.Scripting.Interpreter.Interpreter.Run(InterpretedFrame frame) at Microsoft.Scripting.Interpreter.LightLambda.Run3[T0,T1,T2,TRet](T0 arg0, T1 arg1, T2 arg2) at IronPython.Compiler.Ast.CallExpression.Invoke0Instruction.Run(InterpretedFrame frame) at Microsoft.Scripting.Interpreter.Interpreter.Run(InterpretedFrame frame) at Microsoft.Scripting.Interpreter.LightLambda.Run2[T0,T1,TRet](T0 arg0, T1 arg1) at System.Dynamic.UpdateDelegates.UpdateAndExecute3[T0,T1,T2,TRet](CallSite site, T0 arg0, T1 arg1, T2 arg2) at get_page$509(Closure , PythonFunction , Object ) Traceback (most recent call last): File "C:\Users\sdsds\AppData\Roaming\cYo\ComicRack\Scripts\Comic Vine Scraper\scrapeengine.py", line 382, in _ScrapeEnginescrape_book File "C:\Users\sdsds\AppData\Roaming\cYo\ComicRack\Scripts\Comic Vine Scraper\db.py", line 222, in query_issue File "C:\Users\sdsds\AppData\Roaming\cYo\ComicRack\Scripts\Comic Vine Scraper\cvdb.py", line 454, in _query_issue File "C:\Users\sdsds\AppData\Roaming\cYo\ComicRack\Scripts\Comic Vine Scraper\cvconnection.py", line 144, in _query_issue_details_dom File "C:\Users\sdsds\AppData\Roaming\cYo\ComicRack\Scripts\Comic Vine Scraper\cvconnection.py", line 231, in __get_dom File "C:\Users\sdsds\AppData\Roaming\cYo\ComicRack\Scripts\Comic Vine Scraper\cvconnection.py", line 188, in __get_dom

we'll retry scraping this book again at the end.

======> scraping next comic book: 'Hollywood Zombie Apocalypse v2014 #001 (2014-11).cbz' searching for series that match 'Hollywood Zombie Apocalypse'... ERROR OCCURRED CONTACTING COMICVINE. RETRYING... ------------------- PYTHON ERROR ------------------------ Caught DatabaseConnectionError: Comic Vine database could not be reached url: http://comicvine.com/api/search/?api_key=...&client=cvscraper&format=xml&limit=100&resources=volume&field_list=name,start_year,publisher,id,image,count_of_issues&query=hollywood%20AND%20zombie%20AND%20apocalypse CAUSE: System.Net.WebException: The server committed a protocol violation. Section=ResponseHeader Detail=CR must be followed by LF at System.Net.HttpWebRequest.GetResponse() at Microsoft.Scripting.Interpreter.FuncCallInstruction2.Run(InterpretedFrame frame) at Microsoft.Scripting.Interpreter.Interpreter.Run(InterpretedFrame frame) at Microsoft.Scripting.Interpreter.LightLambda.Run3[T0,T1,T2,TRet](T0 arg0, T1 arg1, T2 arg2) at System.Dynamic.UpdateDelegates.UpdateAndExecute2[T0,T1,TRet](CallSite site, T0 arg0, T1 arg1) at Microsoft.Scripting.Interpreter.FuncCallInstruction5.Run(InterpretedFrame frame) at Microsoft.Scripting.Interpreter.Interpreter.Run(InterpretedFrame frame) at Microsoft.Scripting.Interpreter.LightLambda.Run3[T0,T1,T2,TRet](T0 arg0, T1 arg1, T2 arg2) at IronPython.Compiler.Ast.CallExpression.Invoke0Instruction.Run(InterpretedFrame frame) at Microsoft.Scripting.Interpreter.Interpreter.Run(InterpretedFrame frame) at Microsoft.Scripting.Interpreter.LightLambda.Run2[T0,T1,TRet](T0 arg0, T1 arg1) at get_page$509(Closure , PythonFunction , Object ) Traceback (most recent call last): File "C:\Users\sdsds\AppData\Roaming\cYo\ComicRack\Scripts\Comic Vine Scraper\scrapeengine.py", line 143, in scrape File "C:\Users\sdsds\AppData\Roaming\cYo\ComicRack\Scripts\Comic Vine Scraper\scrapeengine.py", line 257, in _ScrapeEnginescrape File "C:\Users\sdsds\AppData\Roaming\cYo\ComicRack\Scripts\Comic Vine Scraper\scrapeengine.py", line 465, in _ScrapeEnginescrape_book File "C:\Users\sdsds\AppData\Roaming\cYo\ComicRack\Scripts\Comic Vine Scraper\scrapeengine.py", line 777, in _ScrapeEnginequery_series_refs File "C:\Users\sdsds\AppData\Roaming\cYo\ComicRack\Scripts\Comic Vine Scraper\db.py", line 156, in query_series_refs File "C:\Users\sdsds\AppData\Roaming\cYo\ComicRack\Scripts\Comic Vine Scraper\cvdb.py", line 142, in _query_series_refs File "C:\Users\sdsds\AppData\Roaming\cYo\ComicRack\Scripts\Comic Vine Scraper\cvdb.py", line 182, in query_series_refs File "C:\Users\sdsds\AppData\Roaming\cYo\ComicRack\Scripts\Comic Vine Scraper\cvconnection.py", line 54, in _query_series_ids_dom File "C:\Users\sdsds\AppData\Roaming\cYo\ComicRack\Scripts\Comic Vine Scraper\cvconnection.py", line 231, in get_dom File "C:\Users\sdsds\AppData\Roaming\cYo\ComicRack\Scripts\Comic Vine Scraper\cvconnection.py", line 188, in __get_dom

wrote debug logfile: cvs-debug-log-2015-09-01.txt

[crullx]

UPDATE! I just installed the same ComicRack version and same ComicVine Scraper version on my wife's laptop. I used the same settings, the same API and our IP is the same too. On his laptop the scraper runs perfectly! So the IP and API key are not banned. I tried on my laptop again, it doesn't work. I checked my firewall. Comicrack is allowed for both public and private networks. Still it doesn't work on my laptop. Do you have any idea why? Yesterday it was still working, I was using it, then suddenly it stopped working. I reinstalled ComicRack, the scraper too... even Windows too! Nothing helped!

[cbanack]

I just tried the same search that you tried and it is working fine for me. So there is definitely something strange going on. The log file suggests that the Comic Vine server is sending you a blank response when you search for comics, which might be their way of banning you. It might also be caused by something on your end that is stopping the data from getting to Comic Vine Scraper. Though if it's not your firewall, I'm not sure what it would be.

If you visit http://api.comicvine.com on your web browser, does that work?

[crullx]

"If you visit http://api.comicvine.com on your web browser, does that work?"

Yes it works all right. Also if I click "Web" in ComicRack when I selected a comic, it immediately goes to ComicVine website. So the ComicRack program is not disabled in my firewall, I guess. Only the scraper does not work.

[cbanack]

Hmmm...I'm running out of ideas. To double check that the API is truly working, you could try copying and pasting the following URL into your browser, but replace "YOUR_API_KEY" with your api key:

http://comicvine.com/api/search/?api_key=YOUR_API_KEY&client=cvscraper&format=xml&limit=100&resources=volume&field_list=name,start_year,publisher,id,image,count_of_issues&query=hollywood%20AND%20zombie%20AND%20apocalypse

But that's probably still going to work. If it doesn't return any information, then you could try contacting Comic Vine and asking them if they've banned your api, citing that the URL above returns no data.

So then it's a matter of thinking back to what has changed on your computer recently. New programs installed or uninstalled? Seemingly unrelated hardware changes? I haven't heard any complaints from anyone else, so it's got to be something on your end that has changed. That application log is pretty unambiguous, too--it's getting zero data back from the server....

[kino13]

Hello,

Sorry to intrude, but you should seriously check the following: http://comicrack.cyolito.com/forum/8-help/38942-comicvinescrapper-cannot-access-online-database

There have been changes in the comic vine site, they are heavily restricting the usage of the database, most probably the issue is related to comicrack directly and not the plugin, but in any case there is the info received. Please take a look, and thanks a lot for your work.

[cbanack]

Duplicate of issue 421.