Fetching data not working anymore with IMDB

[miziomenca]

Help. It seems that fetching data from imdb is completely down.

[jbjoret]

It has been about a week that it is not working for me either. :'(. I found the following in the httpClient.log ... Host: www.imdb.com User-Agent: Mozilla/4.0 (compatible; MSIE 5.5; Windows 98) Accept-encoding: gzip Connection: Close

HTTP/1.1 302 Found Date: Sat, 21 Apr 2018 13:29:06 GMT Server: Server Location: https://www.imdb.com/find?q=Meet Content-Length: 216 Keep-Alive: timeout=2, max=20 Connection: Keep-Alive Content-Type: text/html; charset=iso-8859-1

GET /find?q=Meet HTTP/1.0 Host: www.imdb.com User-Agent: Mozilla/4.0 (compatible; MSIE 5.5; Windows 98) Accept-encoding: gzip Connection: Close_

It looks like IMDB redirects to https, and that the client can't handle it.

[kec2]

I can't reach imdb.com at all.

[kec2]

I just tryed: https://www.imdb.com/find?s=all&q=ready+player+one

result: D'oh! We're sorry, something went wrong.

Please try again...wait...wait...yep, try reload/refresh now.

But if you are seeing this again, please report it here.

Please explain which page you were at and where on it that you clicked

Thank you!

[andig]

I have not actively been maintaining videodb for a couple of years now (actually: it could use a rewrite on modern technology like Symfony and Bootstrap) but I'd be happy to accept PRs.

[jbjoret]

IMDB page is working. But as I said, now it redirects to https which the httpClient.php cannot handle. Using Bootstrap could be cool :). @andig sorry for my ignorance but what does PRs mean ? :)

[andig]

@jbjoret PR is a pull request- a contribution to this repo that you can make

as for httpClient it should be entirely replaced, e.g. by https://github.com/guzzle/guzzle. Guzzle v6 still supports PHP 5.5 so it should be pretty much plug and play.

[kec2]

@andig Do you have any advice of how to "attack" the task of plugging Guzzle into VideoDB? I guess that lib/ is the place to put Guzzle.

[andig]

I would start moving to composer: composer install it and add the /vendor folder to the repo. In httpclient inlcude vendor/autoload.php and use the guzzle classes. As I‘ve said- the codebase is severely outdated :(

[miziomenca]

I'm so sorry guys but i'm not a coder. I tried to study and understand Guzzle but.... sorry. My entire collection (>2400 movies) is on my VideoDB forever.

[kec2]

I can try and look at it but I'm not making any promises.

[kec2]

I have upgraded httpclient.php with Guzzle. But i'm not able to create a pull request. I'm not sure if it is missing rights or skills. I'm using the GitHub desktop tool.

[andig]

You can email me your folder.

Viele Grüße, Andreas

Am 27.04.2018 um 21:27 schrieb kec2 notifications@github.com:

I have upgraded httpclient.php with Guzzle. But i'm not able to create a pull request. I'm not sure if it is missing rights or skills. I'm using the GitHub desktop tool.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

[kec2]

Hi Andreas

The zip only contains files that I have modified or added.

Regards

Klaus Christiansen

---

From: andig notifications@github.com Sent: 27 April 2018 22:13 To: andig/videodb Cc: kec2; Comment Subject: Re: [andig/videodb] Fetching data not working anymore with IMDB (#96)

You can email me your folder.

Viele Grüße, Andreas

Am 27.04.2018 um 21:27 schrieb kec2 notifications@github.com:

I have upgraded httpclient.php with Guzzle. But i'm not able to create a pull request. I'm not sure if it is missing rights or skills. I'm using the GitHub desktop tool.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

— You are receiving this because you commented. Reply to this email directly, view it on GitHubhttps://github.com/andig/videodb/issues/96#issuecomment-385081634, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AC9y9jPAP3lpjKqZJjv5Mzo_Sk759MhZks5ts3vPgaJpZM4TeXlj.

[andig]

Can you email me at cpuidle@gmx.de? No zip file received...

[andig]

I've added a quick PR based on initial ideas by @kec2. On a quick test I could add a movie from IMDB.

This PR is most likely not fully working and will need further attention. If anybody wants to contribute please send PRs through github.

Note: the PR adds the vendor folder which is typically not the way to use composer dependency management. It was only done in this case to not introduce another dependency on composer.

For a more full-fledged rewrite of videoDB we'd need to find a sponsor as I'm busy with other projects and no longer use videoDB myself.

[olaf-d-aix]

Hi Andreas, thanks for the quick fix. But it seems not to be complete. Some data from IMDB can now be fetched again, mainly the pictures. If I open a movie right now, updating the actor images works again. Formerly it took bloody ages until a timeout occurs, so videodb hangs for a while. But I'm not able to add a new movie! If I try that I get an error: "Server returned wrong status." Is a simple "git pull" enough?

[andig]

You‘ll need to merge the pr for testing or checkout it‘s branch.

Viele Grüße, Andreas

Am 30.04.2018 um 15:39 schrieb olaf-d-aix notifications@github.com:

Hi Andreas, thanks for the quick fix. But it seems not to be complete. Some data from IMDB can now be fetched again, mainly the pictures. If I open a movie right now, updating the actor images works again. Formerly it took bloody ages until a timeout occurs, so videodb hangs for a while. But I'm not able to add a new movie! If I try that I get an error: "Server returned wrong status." Is a simple "git pull" enough?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

[olaf-d-aix]

Okay, I did a "git checkout guzzle". Now I get no longer the "Server returned wrong status." error, but now I get a blank white screen in the browser window and nothing happens. I put a "error_reporting(E_ALL);" in the lookup.php to do a little debugging, but I still can see nothing.

[olaf-d-aix]

Finally I managed to get an "An exception occured: Server error: GET http://www.imdb.com/find?q=test resulted in a 503 Service Unavailable response:" error.

[kec2]

It seams that engines/*.php has not been updated. Also the url to IMDB should use https.

[miziomenca]

So… sorry, i realy don't wanna be annoying but… nobody found a solution? There's no way? I'm so sad for my inability to bring support about the problem.

[kec2]

You can take a look of the fork I made: https://github.com/kec2/videodb Remember to make a backup current videodb installation.

[andig]

I've updated https://github.com/andig/videodb/pull/97 with the engine changes by @kec2

@kec2 those were not mergeable and instead basically replaced the old files. Not sure why (e.g. imdb). Please check if this works for you. In not I'd be happy to accept a PR to readme that this repo is no longer maintained and point to your.

[kec2]

Hi Andig,

I'm confident that imdb.php works. Maybe the fact the some files can't be merged is due to space/tab and/or different line endings.

The changes to the other engines are due to changes in httpClient.php. I have never used the other engines then IMDB, so I don't know if they work or not.

Hilsen

Klaus Christiansen

---

From: andig notifications@github.com Sent: 21 May 2018 19:38 To: andig/videodb Cc: kec2; Mention Subject: Re: [andig/videodb] Fetching data not working anymore with IMDB (#96)

I've updated #97https://github.com/andig/videodb/pull/97 with the engine changes by @kec2https://github.com/kec2

@kec2https://github.com/kec2 those were not mergeable and instead basically replaced the old files. Not sure why (e.g. imdb). Please check if this works for you. In not I'd be happy to accept a PR to readme that this repo is no longer maintained and point to your.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://github.com/andig/videodb/issues/96#issuecomment-390727230, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AC9y9v_cTMgtQ37o1FHPyyY1DdMiidPnks5t0vuggaJpZM4TeXlj.

[andig]

@kec2 seems it was the line endings. I've updated once again so it looks like a proper merge and would appreciate a test. All your changes plus some fixes to the http client should be inlcuded now. I've only done a superficial test though.

[copperhead57]

can anyone help - using 'GUZZLE' branch from andig and getting the following error after clicking imdb online tab. An exception occurred: cURL error 60: SSL certificate problem: unable to get local issuer certificate (see http://curl.haxx.se/libcurl/c/libcurl-errors.html) Array ( [0] => Array ( [function] => exception_handler [file] => D:\web\wamp64\www\videodb-guzzle\core\functions.php [line] => 0 [params] => Array ( ) ) [1] => Array ( [function] => errorpage [file] => D:\web\wamp64\www\videodb-guzzle\core\functions.php [line] => 99 [params] => Array ( ) ) )

seems to be coming from file - D:\web\wamp64\www\videodb-guzzle\vendor\guzzlehttp\guzzle\src\Handler\CurlFactory.php - line 176.

using php - 5.6.35. thanks Paul.

[andig]

@copperhead57 what are you trying to fetch when that happens?

The remote site might be using a CA that php 5.6 didn‘t know yet. So either upgrade php/curl or disable certificate verification in httpclient, see https://stackoverflow.com/questions/28066409/how-to-ignore-invalid-ssl-certificate-errors-in-guzzle-5 for how to.

[copperhead57]

@andig i am just clicking the imdb online tab to get the first screen displayed.

will review look into php/curl etc tomorrow

[copperhead57]

@andig success (well mostly)

• firstly Curl error 60 fixed with following steps
  • ensure ssl_module in apache is active
  • ensure php_openssl is active
  • download cacert.pem from https://curl.haxx.se/docs/caextract.html to directory D:\web\wamp64\bin\php\php5.6.35\extras\ssl\
  • update php.ini file [openssl] with openssl.cafile= "D:\web\wamp64\bin\php\php5.6.35\extras\ssl\cacert.pem"
  • restart all services. have been able to add new movie from pulling from IMDB from the 'new' tab

[copperhead57]

moving this to its own issue. (with possible solution). When clicking the 'imdb online' tab data is being returned but all the css formating seems missing