search

cbanack / comic-vine-scraper

An add-on script for ComicRack that lets you copy details from Comic Vine into your comic books.
243 stars 47 forks source link

The Comic Vine online database could not be reached #496

Closed EndorEwoksFC closed 6 days ago

EndorEwoksFC commented 1 week ago

Issue #492 looks like it pertains to older versions of Windows but I'm running Windows 11 and am still getting this error. I have tried disabling my firewall but the error still persists. It only seems to show itself when the scraper has returned multiple results and I select a series to match it to. It will think for a few seconds and then come up with the error.

I also checked my Comic Vine API and verified that there is no issue there.

Previous issue #492 mentions pressing Ctrl + Shift + L when the error dialog box is open to be able to print the logs but that is working for me (nor is Ctrl + L or Ctrl + Alt + L).

cbanack commented 1 week ago

There are many things that could be causing the problem in #492, basically anything that interrupts the scrapers ability to connect to ComicVine. If you're able to connect to the API in your browser, then that rules out a lot of them.

Ctrl + Shift + L should work to get you a log file -- just make sure the scraper dialog window has keyboard focus when you press that key combo (i.e. click somewhere on the scraper dialog right before pressing Ctrl + Shift + L). The log file should have a good clue as to what is going wrong.

EndorEwoksFC commented 1 week ago

Sorry, new to all of this ComicRack and ComicVine stuff...if by "connect to the API in the browser" you just mean that I'm able to get to my API page and see that it shows my usage and that I'm in good standing, then yes it looks good.

As for getting the log, when I press Ctrl + Shift + L all I get is the Windows error sound. Nothing opens and no log is added to my clipboard or anything.

KaeAlSa commented 1 week ago

Same issue here. I'm on windows 11. Updated CRCE and used the latest scraper version.

@EndorEwoksFC I couldn't get the log during the error either, I had to do it when the prompt came up saying Scraped details for 0 comic books skipped 1 comic book.

@cbanack it does seem to pull in the info from Comic Vine. I see my API requests go up, I set it to manually confirm and it populates the list of possible issues, I can select the match I want then it goes to the next step. It just seems to be an issue when it's saving, or pulling the final info for the specific comic? Below is my log.

cvs-debug-log-2024-06-21.txt

cbanack commented 6 days ago

@EndorEwoksFC: you really should be able to save an error log, this is a feature that has been around for over a decade in the scraper, and it hasn't been changed for years. And without that log, I really have no way of telling you why the scraper is not working. Perhaps you have some other tool installed in Windows that is capturing that keypress? Otherwise, you just have to make sure a Comic Vine Scraper window has the keyboard focus (i.e. click on it) before pressing the hotkeys. And remember -- Comic Vine Scraper is NOT the same thing as ComicRack. One of the Comic Vine Scraper dialog windows needs to have focus, NOT the ComicRack window or any other window that you might have open.

(ComicVineScraper is just a plugin that I wrote about 15 years ago for the ComicRack program, which is a separate thing that is built and maintained by someone else.)

@KaeAlSa: your log tells me that the scraper was unable to reach the ComicVine database. This can sometimes happen because the people who run that database made a mistake and it is just down -- I've seen it go down for a day or even two, and then it comes back up and things start working again.

It's also possible that you are behind a firewall or some other network thing that is making your computer unable to reach the Comicvine API. A simple way that either of you could test this is to vist the following URL in your browser (note, you must replace <APIKEY> with your actual API key):

https://comicvine.gamespot.com/api/issues/?api_key=<APIKEY>&client=cvscraper&format=xml&filter=volume:101582

You should see response page with a bunch of XML code, i.e. something that isn't a big empty error page. If there is an error message, or the page just hangs there without loading, then there is something about your network that is preventing your computer from talking to ComicVine, and you'll need to fix that before the scraper can work.

KaeAlSa commented 6 days ago

@cbanack I figured out my issue. It is a rate limit but not one that was clear to me.

image

At the bottom of the API usage log it'll still say your request rate is fine, but I noticed I'd run into issues when an individual request type jumps above 200.

image

Actually reading the terms of service indicated I was right, even though the rate request was fine, it still was rate limiting me after hitting 200 on one.

So I've just been doing a few at a time. It'll still hit the limit if I have a bunch selected even before letting me manually select issues so it effectively scrapes 0 comics.

EndorEwoksFC commented 6 days ago

Looks like I was having the same issue as @KaeAlSa because my API page was showing the same thing. I just assumed that "Your request rate is fine" meant that I was good to go.

As far as the error log, as @KaeAlSa also said earlier, it looks like I had to hit the Ctrl + Shift + L on the prompt following the error message.