ComicVine Stalls then Crashes

[GoogleCodeExporter]

DESCRIBE THE PROBLEM:

Comic Vine scrapper crashes on amazing spider-man series 1, issues 200 or 
higher. It happens each time.

WHAT STEPS WILL REPRODUCE THIS PROBLEM? (Please include the exact name of
the eComic book that you were trying to scrape, if possible.)
1. select spider-man series described above
2. automation -> comicvine scrapper
3. start scrapping

WHAT VERSION OF COMICVINESCRAPER ARE YOU USING?

v1.0.25

PLEASE PROVIDE ANY ADDITIONAL INFORMATION THAT MAY BE OF USE

Original issue reported on code.google.com by mgjsc...@gmail.com on 15 Jul 2010 at 8:58

Attachments:

• comicrack_crash

[GoogleCodeExporter]

Thank you for the bug report.

I followed your instructions, and I was able duplicated the crash myself.   It 
looks to me like the ComicVine website (where all the data for the scraper 
comes from) is having some technical problems right now, especially with series 
(Spiderman, Batman, 2000 AD) that have a large number of issues.

This problem may just be temporary (try again later!) or it may be the result 
of some permanent change that ComicVine made, in which case I will have to try 
to find a way to work around it.

The scraper should start working properly again as soon as the following link 
(in a web browser) consistently works without error:

http://api.comicvine.com/volume/2127/?api_key=4192f8503ea33364a23035827f40d415d5
dc5d18&format=xml&field_list=issues

That's the address of the data the the scraper is trying to retrieve, but right 
now when I run that link directly from my browser, it doesn't work. :(

Original comment by cban...@gmail.com on 15 Jul 2010 at 9:21

[GoogleCodeExporter]

I just tried this again, and it looks like ComicVine has sorted out their 
issues and things are working again.   Can you let me know if this problem has 
fixed itself on your end, too?

Original comment by cban...@gmail.com on 16 Jul 2010 at 2:46

[GoogleCodeExporter]

I am getting a the same problem trying to scrape Batman 701.  
Log shows a parsing error at that same link

Original comment by cvluc...@gmail.com on 26 Jul 2010 at 5:30

Attachments:

• log.txt

[GoogleCodeExporter]

@cvlucero:

I imagine this has resolved itself for you by now, since it looks like that 
link is working ok again...

The problem is on ComicVine's end, but since it looks like it may continue to 
happen now and then, I'll have to see if I can find a way to work around it.

Original comment by cban...@gmail.com on 27 Jul 2010 at 7:19

[GoogleCodeExporter]

Issue 118 has been merged into this issue.

Original comment by cban...@gmail.com on 27 Jul 2010 at 6:10

[GoogleCodeExporter]

Issue 119 has been merged into this issue.

Original comment by cban...@gmail.com on 28 Jul 2010 at 2:15

[GoogleCodeExporter]

I have reported this problem to ComicVine:

http://www.comicvine.com/forums/bug-reporting/2/comicvine-api-truncates-response
s-for-large-volumes/561712/

I am also looking for other ways that I can work around their bug.

Original comment by cban...@gmail.com on 28 Jul 2010 at 3:22

• Added labels: Priority-Critical
• Removed labels: Priority-Medium

[GoogleCodeExporter]

Original comment by cban...@gmail.com on 28 Jul 2010 at 3:23

• Changed title: ComicVine Stalls then Crashes

[GoogleCodeExporter]

Original comment by cban...@gmail.com on 28 Jul 2010 at 3:23

• Changed state: Accepted

[GoogleCodeExporter]

The only way to directly fix this bug is for the ComicVine team to fix their 
API/website so that really large queries (like one mentioned in comment 1) will 
work properly (instead of being truncated.)

However, I am not holding my breath on any action from their side.

Instead, I have reworked how the scraper obtains issue data (as if version 
1.0.26), which basically avoids the really big query altogether.   So this bug 
should no longer occur.

Original comment by cban...@gmail.com on 30 Jul 2010 at 6:48

• Changed state: Fixed

[GoogleCodeExporter]

Fixed for 1.0.26

Original comment by cban...@gmail.com on 30 Jul 2010 at 7:33