Youtube with https - Githubissues

VoidAndAny commented 3 years ago

Your lib is used in Bolts CMS (at least 3.x version) to embed Youtube video. Since ?? weeks/months, it doesn't work anymore with the following error :

[HTTP/1.1 500 Internal Server Error 756ms]
    success false
    code    500
    error   Object { type: "\\Client", file: "vendor/guzzlehttp/guzzle/src/Exception/RequestException.php", line: 113, … }
    type    "\\Client"
    file    "vendor/guzzlehttp/guzzle/src/Exception/RequestException.php"
    line    113
    message "Client error: `GET http://www.youtube.com/oembed?url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3D-7mdBNjaMqc&format=json` resulted in a `403 Forbidden` response:\n{\n \"error\": {\n \"code\": 403,\n \"message\": \"SSL is required to perform this operation.\",\n \"status\": \"PERMISSION_D (truncated...)\n"

Can we access Youtube URL with https : https://www.youtube.com/oembed

(I made the correction in my vendors and it's OK)

I can make a PR if you want.

Tahnks

kinglozzer commented 3 years ago

I’ve encountered this too, admittedly only over the last couple of days. Is there any reason why some providers use HTTP instead of HTTPS? PR: https://github.com/oscarotero/Embed/pull/418

cleentfaar commented 3 years ago

Okay so I've done some digging and found the likely issue.

There are two ways used by Embed to determine the correct embed-endpoint of a given provider.

First it tries to visit the video URL itself and see if there are any <link> tags that indicate the correct URL to use. For example, on a rickroll video you will see this:

<link rel="alternate" href="android-app://com.google.android.youtube/http/www.youtube.com/watch?v=oHg5SJYRHA0">
<link rel="alternate" href="ios-app://544007664/vnd.youtube/www.youtube.com/watch?v=oHg5SJYRHA0">
<link rel="alternate" type="application/json+oembed" href="http://www.youtube.com/oembed?format=json&amp;url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3DoHg5SJYRHA0" title="RickRoll&#39;D">
<link rel="alternate" type="text/xml+oembed" href="http://www.youtube.com/oembed?format=xml&amp;url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3DoHg5SJYRHA0" title="RickRoll&#39;D">

Note that youtube itself is providing an incorrect protocol in it's uris (http instead of https)! Problem number one found!

Okay, I thought, so YouTube is messing up... But why is Embed even trying these link tags when it has preconfigured providers (Youtube, Vimeo etc) that have been designed to do this as indicated here: https://github.com/oscarotero/Embed/blob/master/src/resources/oembed.php#L932.

The answer to that can be found here: https://github.com/oscarotero/Embed/blob/master/src/OEmbed.php#L56-L60. As you can see there, the detectEndpointFromProviders is only used as a last resort for some reason.

So summarizing:

Youtube is serving a bad url (http protocol) in it's alternate <link> tags on video pages
Embed always tries <link> tags first before using it's built-in providers. If the provider was used it would have ignored youtube's error and use the proper (https) protocol by default.

Now to the actual fix, that's a tougher nut to crack. Here are the options as I see them, with my opinion added:

Let Embed always give priority to it's providers, before trying <link> tags:
- PRO: nice for us devs because it fixes the issues we have now embedding youtube video's, without having to wait for youtube to notice.
- CON: You put even more trust in Embed's pre-configured providers, rather than letting these videosites communicate their own URIs. Not a big problem short-term, but hard to maintain long-term.
Wait for YouTube to fix their <link rel="alternate"> tags:
- PRO: fix problems at the source rather than working around them through this Embed package.
- CON: Waiting for YouTube to change something, whilst having customers complaining that they cant watch embedded videos on our website.

In the meantime my company can't afford to break embedded videos so I will work with a fork until the package author makes a decision on this.

Hopes this helps others that are trying to debug this.

oscarotero commented 3 years ago

Ok, I see. Thanks for the valuable info @cleentfaar

I'm going to wait for Youtube. Changing the priority of the detectors may fix this issue but open others. The philosophy of this library is rely first in the data provided by the site (that can change anytime) and only use the pre-configured data as the last resort, because it can become deprecated anytime. Note also that the embed code is returned even if the oembed endpoint is failing (as you can see in the demo: https://oscarotero.com/embed/demo/index.php?url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3DoHg5SJYRHA0&settings=) This is due it's using the video url provided in the og:video:url meta element.

cleentfaar commented 3 years ago

@oscarotero not sure I understand what you mean, the demo you link to doesnt work either?

oscarotero commented 3 years ago

What I mean is that, even if the oEmbed endpoint fails, we can get a embed code:

because Embed get the video url from opengraph:

cleentfaar commented 3 years ago

ok, but thats not what happening in my code at the moment, it just returns null, what should I change to use that opengraph?


        $info = $embed->get($url);
        $oembed = $info->getOEmbed();
        $embedHtml = $oembed->get('html');

are you suggesting doing something with $info->getMeta() instead of $info->getOEmbed(), there's no single method that I can use which does this for me?

oscarotero commented 3 years ago

Sorry, I don't understand. Doesn't this code work for you?

$info = $embed->get($url);
$embedHtml = $info->code;

I mean, you don't have to get the different apis directly (oembed, meta, etc). Embed library does it automatically for you, as you can see in the example: https://github.com/oscarotero/Embed#usage

leevigraham commented 3 years ago

@oscarotero to add some further clarification:

This works fine:

$info = $embed->get('https://www.youtube.com/watch?v=sRy8aLzZRyA&feature=youtu.be');
$embedHtml = $info->code;

This fails:

$info = $embed->get('https://www.youtube.com/watch?v=sRy8aLzZRyA&feature=youtu.be');
$embedHtml = $info->getOEmbed()->all();

with (json output):

"ombed": {
        "error": {
            "code": 403,
            "message": "SSL is required to perform this operation.",
            "status": "PERMISSION_DENIED"
        }
    }

Following the url directly in the browser: http://www.youtube.com/oembed?format=json&url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3DsRy8aLzZRyA performs a 301 redirect. I wonder if the underlying code is not following redirects?

gabrielsolomon commented 3 years ago

I tried to fix this in #419

oscarotero commented 3 years ago

The function $info->all(); does not exist in this library.

leevigraham commented 3 years ago

Sorry that should have been $info->getOEmbed()->all(); updated my comment.

oscarotero commented 3 years ago

Ok. This is because this library not only use oEmbed API to get the data, but also html meta tags, linked data, etc. In this case, due oEmbed endpoint fails, the data is returned by opengraph or twitter cards. You can see the full code detector here: https://github.com/oscarotero/Embed/blob/master/src/Detectors/Code.php

You shouldn't use the oEmbed class directly, but the $info->code property.

oscarotero / Embed

Youtube with https #417