Lot of ERROR com.moviejukebox.tools.WebBrowser

[Omertron]

Original issue 2552 created by Omertron on 2012-12-22T18:30:13.000Z:

What steps will reproduce the problem?

1. run yamj with empty jukebox
2. Idem for each rerun Yamj

There is this message for all movie :

WebBrowser: Error getting URL http://www.google.com/search?q=Quand+je+serai+peti t+%282012%29+site%3Awww.imdb.com&meta=, Server returned HTTP response code: 503 for URL: http://www.google.com/sorry/?continue=http://www.google.com/search%3Fq% 3DQuand%2Bje%2Bserai%2Bpetit%2B(2012)%2Bsite%253Awww.imdb.com%26meta%3D

I doing some search in new feature "PriorityChecks" . But I haven't find.

before I had these messages with a empty jukebox but less. Then I did not when I rerun YAMJ.

I don't understand what is searched on Google. I don't use google. This take lot a of time when I rerun Yamj: I do not recover any more pictures of people. Perhaps these two problems are related.

My setting in moviejukebox.properties for scanner :

... poster.scanner.SearchPriority.movie=allocine poster.scanner.SearchPriority.tv=thetvdb mjb.internet.plugin=com.moviejukebox.plugin.AllocinePlugin mjb.internet.tv.plugin=com.moviejukebox.plugin.TheTvDBPlugin mjb.internet.person.plugin=com.moviejukebox.plugin.ImdbPlugin ...

Windows 8 x64 + Java Yamj 2.8 R3399 + eversion r0179 + evZap 1.2.1 + Mod People 5.3 Fr

PS : In all case thanks all for your big work and your time

[Omertron]

Comment #1 originally posted by Omertron on 2012-12-22T20:01:23.000Z:

Sorry, you can close this invalid issue All works now. Pb internet I think Next time I will wait more before to post issue :/

[Omertron]

Comment #2 originally posted by Omertron on 2012-12-22T22:37:29.000Z:

Just for information

this bug is back. So if I understand google is use for find imdb url or something like that.

If I use the url return in message error Google asks me to verified if I m not a robot because there is a lot off trafic from my ip adress:

I hope google not change its security policy

[Omertron]

Comment #3 originally posted by Omertron on 2012-12-22T22:45:25.000Z:

google message: "Our systems have detected unusual traffic on your network. This page allows you to verify that this is really you sending the requests, and not a robot"

[Omertron]

Comment #4 originally posted by Omertron on 2012-12-23T20:15:11.000Z:

This happens after the approximately 170 first movie scanned with YAMJ. I understand there is no fix Yamj for that. But I m very curious to know if there are other people in this case.

[Omertron]

Comment #5 originally posted by Omertron on 2012-12-23T20:18:49.000Z:

Hi Nicolas, the issue 2553 has just been fixed, so now a valid IMDB result should be returned again, so that less google requests are needed.

Please use the latest snapshot and check, if there are any problems with google or too much google requests.

Google will only be requested, if there is no valid IMDB id found on the IMDB site. Cause IDMB changed to layout of the search results, there couldn't be determined a valid ID from IMDB until vincent.ysmal has fixed issue 2553.

[Omertron]

Comment #6 originally posted by Omertron on 2012-12-24T00:13:18.000Z:

I have run Yamj r3411 with empty jukebox and 1420 movie in library. There is more than 500 error in moviejukebox.ERROR.log about "Server returned HTTP response code: 503 for ..." :/

[Omertron]

Comment #7 originally posted by Omertron on 2012-12-24T12:07:56.000Z:

As I unstand the google message, than your IP is somehow blocked for gooogle search until the "too much requests" ends.

At the moment I don't know a workaround; perhaps using an anonym proxy will help (proxy settings could be set in YAMJ).

Perhaps the project owner of YAMJ might find an agreement with google to use a client string for requests which will not be blocked.

[Omertron]

Comment #8 originally posted by Omertron on 2012-12-24T12:42:54.000Z:

ok thanks I can live with that. this is happend then I launch Yamj with empty Jukebox. It is not all days. I do that when there is a major enhancement yamj.

There is this issue 2515 who can help to win time with big library and empty jukebox. I try my luck. :)

[Omertron]

Comment #9 originally posted by Omertron on 2012-12-24T13:24:34.000Z:

Usually the web page or google will allow you to override and "prove" you are not a robot (You actually are, because it's YAMJ doing the scraping)

Do that, and it should allow you to restart scanning

[Omertron]

Comment #10 originally posted by Omertron on 2012-12-24T14:05:37.000Z:

Thank you very much :)

For the others persons in this case this is the link for do :

http://support.google.com/websearch/bin/request.py?hlrm=en&contact_type=ban&&hl=en

[Omertron]

Comment #11 originally posted by Omertron on 2012-12-28T01:31:56.000Z:

Form google did not effect. Google applies no change. (I have a fixed IP address). After 1 day without Yamj requests IP address is still blocked after around 350 films. it is very frustrating not to be able to change this. :(

What are the consequences for YAMJ. This is only the information about people which is incomplete ? I m not sure :/

I use imdb plugin only for person:

This is my config for scrapper in moviejukebox.properties

mjb.internet.person.plugin=com.moviejukebox.plugin.ImdbPlugin poster.scanner.SearchPriority.movie=allocine poster.scanner.SearchPriority.tv=thetvdb mjb.internet.plugin=com.moviejukebox.plugin.AllocinePlugin mjb.internet.tv.plugin=com.moviejukebox.plugin.TheTvDBPlugin

I will make a test with: mjb.MaxThreadsProcess=5 (instead of mjb.MaxThreadsProcess=20)

thanks

[Omertron]

Comment #12 originally posted by Omertron on 2012-12-28T14:51:15.000Z:

Modmax is right. The only way is to use a proxy settings in YAMJ.

[Omertron]

Comment #13 originally posted by Omertron on 2012-12-28T16:17:23.000Z:

No I m wrong. It is always the same thing. IP proxy is banned after 350 Movies.

"Google will only be requested, if there is no valid IMDB id found on the IMDB site".

All my movies are corrected named with date. I don't understand why so much no valid IMDB id.

All of this make me crazy.

I will open a new issue.