search

oscarotero / Embed

Get info from any web service or page
MIT License
2.09k stars 312 forks source link

Embed on an url work on localhost and demo website, but not in my production server #411

Open AxelBriche opened 3 years ago

AxelBriche commented 3 years ago

Hello,

$embed = new Embed();
$info = $embed->get('http://www.jeuxvideo.com/');

dump($info->image);

This url work on demo website and has an image. This url work on localhost and show me a Slim\Psr7\Uri instance with image infos.

But on my production server, $info->image return null. Why please?

oscarotero commented 3 years ago

This can happens due many reasons. The most common is the site detects the client ip and returns different content. This happened multiple times (sites returning different content if the request is done from different servers).

Try to run the demo page from the production server so you can check the received response.

lozeone commented 3 years ago

I am seeing this happen too, specifically with instagram.

What I discovered was that the initial dispatch of the url in Embed::process() $response = $dispatcher->dispatch($url); returns an error code (429) because the URL does not use the IG endpoint with the access token. (On localhost it passes).

But since the response has an error code, $adapter::check($response) fails and it never gets a chance to check the url with the correct endpoint and token.

I was able to verify this by hacking $response->isValid() to always return true. Which obviously isn't the right thing to do. But once I do that the adapter runs and correctly embeds the post. Not sure where this needs to be fixed.

using v3.4

Hope that helps.

AxelBriche commented 3 years ago

I have installed the demo on my server and I have the same problem:

http://178.62.219.208/index.php?url=https%3A%2F%2Fwww.jeuxvideo.com&settings=

The HTML content is:

<html><body><h1>403 Forbidden</h1>
Request forbidden by administrative rules.
</body></html>

Why 403 Forbidden on my server and no error on the demo website please? The website is hosted by DigitalOcean.

oscarotero commented 3 years ago

I suspect some sites block request from DigitalOcean servers because it's common to use these serves for scrapping. FYI, my server is in Hetzner (https://www.hetzner.com/cloud)

seanBlommaert commented 3 years ago

When I change the following in Embed::extract() it seems to work.

$uri = $this->crawler->getResponseUri($response) ?: $request->getUri();

$uri = $request->getUri();

The getResponseUri($response) method converts the embed URL because instagram sometimes requires a login. Didn't seem to be able to figure out when exactly, me and my colleagues get different results for the same URL, with and without the company VPN.

rupalgohel commented 2 years ago

hi I am facing this same issue, not getting profile image on live/production server. [https://stackoverflow.com/questions/71121676/get-meta-image-of-any-social-media-from-respective-profile-link-php]