Closed i300220 closed 1 week ago
Benjamin-Loison
commented
2 weeks ago The issue only happens on the official instance, I think it is related to #11, hosting your own instance is a workaround, I will investigate such that the official instance at least returns an error.
i300220
commented
2 weeks ago Humm, It used to work very well before. See this post from may 28 2024 - https://github.com/cn-tools/cntools_FreshRssExtensions/issues/8#issuecomment-2127853030
https://github.com/Benjamin-Loison/YouTube-operational-API/issues/11 is 2 years old.
Benjamin-Loison
commented
2 weeks ago I know that it worked before. #11 is just the issue where I gather information about YouTube detecting the instance as automated, hence blocking it, which seems to be the case here as local instances work fine.
i300220
commented
2 weeks ago Right, thanks for the precision. Appreciated.
Benjamin-Loison
commented
1 week ago Viewing in Firefox curl 'https://www.youtube.com/watch?v=n8vmXvoVjZw' result:
I will check if #296 patch still works fine in its context and here.
VIDEO_IDS=('UPrkC1LdlLY' 'a9cyG_yfh1k')
YOUTUBE_OPERATIONAL_API_INSTANCE_URL='https://yt.lemnoslife.com'
for videoId in "${VIDEO_IDS[@]}"
do
echo $videoId
curl -s "$YOUTUBE_OPERATIONAL_API_INSTANCE_URL/videos?part=music&id=$videoId" | jq '.items[0].music.available' -
doneUPrkC1LdlLY
false
a9cyG_yfh1k
trueSource: #296#issuecomment-2278630907
getJSONPathFromKey.py n8vmXvoVjZw '' ytInitialPlayerResponse | grep -i robot 25 /playabilityStatus/reason Connectez-vous pour confirmer que vous n'êtes pas un robot
75 /playabilityStatus/errorScreen/playerErrorMessageRenderer/reason/simpleText Connectez-vous pour confirmer que vous n'êtes pas un robotSame string so let us use /playabilityStatus/reason.
curl -H 'Accept-Language: en' 'https://www.youtube.com/watch?v=n8vmXvoVjZw' > n8vmXvoVjZwSource: blob/c4539bc55d74c556570e1eabb710369a7fca159c/common.php#L178
getJSONPathFromKey.py n8vmXvoVjZw reason ytInitialPlayerResponse25 /playabilityStatus/reason Sign in to confirm you’re not a bot
64 /playabilityStatus/errorScreen/playerErrorMessageRenderer/reasonUsing the following gmail.com email address:
-----BEGIN PGP MESSAGE-----
hF4DTQa9Wom5MBgSAQdAC7CuGL/Gl5Lf1zNXBRNfVifsM5lzeUIeLKxttXYWoFIw
MWMJ7gYqCO/j8MEEN/rvsjlQ8VekA5UMylEYJX3kR6WS91kuCutJ4vNx0+3HQu+7
0k4BmyM0Mrriwx6S9aHr9/YdvVw0CV8WCaIx/zJp0fwF2ppszhTjiYKemX8Cjqfz
0Ddk6C7IG6OlhpW3o3roHIWucI1TubALeoFN55HVXeI=
=+DrJ
-----END PGP MESSAGE-----I do not know which one was used in #296.
minimizeCURL.py curl.sh '"lengthSeconds":"1222"'...
curl 'https://www.youtube.com/watch?v=n8vmXvoVjZw' -H 'Cookie: __Secure-3PSID=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX'grep -r 'file_get_contents(' | grep -v "file_get_contents('tests/"common.php: $result = file_get_contents($url, false, $context);
noKey/index.php: $content = file_get_contents(KEYS_FILE);
noKey/index.php: $response = file_get_contents($realUrl, false, $context);
addKey.php: $keysContent = file_get_contents(KEYS_FILE);
index.php: $keysCount = file_exists(KEYS_FILE) ? substr_count(file_get_contents(KEYS_FILE), "\n") + 1 : 0;
index.php: $ref = str_replace("\n", '', str_replace('ref: ', '', file_get_contents('.git/HEAD')));
index.php: $hash = file_get_contents(".git/$ref");grep -r 'fileGetContentsAndHeadersFromOpts('common.php: function fileGetContentsAndHeadersFromOpts($url, $opts)
common.php: [$result, $headers] = fileGetContentsAndHeadersFromOpts($url, $opts);curl -I 'https://www.youtube.com/watch?v=n8vmXvoVjZw'returns 200 code on my laptop and on the official instance.
grep -rn 'getRemote('common.php:101: function getRemote($url, $opts = [], $verifyTrafficIfForbidden = true)
common.php:131: return json_decode(getRemote($url, $opts, $verifyTrafficIfForbidden), true);
common.php:157: $html = getRemote($url, $opts);
videos.php:373: $html = getRemote("https://www.youtube.com/watch?v=$id");
playlistItems.php:62: $res = getRemote($url, $httpOptions);
addKey.php:25: getRemote($addKeyToInstance . "addKey.php?key=$key&forceSecret=" . ADD_KEY_FORCE_SECRET);grep -rn 'getRemote('common.php:101: function getRemote($url, $opts = [], $verifyTrafficIfForbidden = true)
common.php:131: return json_decode(getRemote($url, $opts, $verifyTrafficIfForbidden), true);
common.php:157: $html = getRemote($url, $opts);
addKey.php:25: getRemote($addKeyToInstance . "addKey.php?key=$key&forceSecret=" . ADD_KEY_FORCE_SECRET);It seems that we can focus on videos and more especially on endpoints requiring the video to load. So the following does not seem necessary, should focus on endpoints potentially suffering of the issue, so YouTube operational API videos endpoint but what parts?
diff --git a/common.php b/common.php
index 50fe0c2..c58f2ef 100644
--- a/common.php
+++ b/common.php
@@ -168,6 +168,8 @@
return getJSONFromHTML($url, $opts, $scriptVariable, $prefix, $forceLanguage, $verifiesChannelRedirection);
}
}
+ // #24 would maybe reduce the following need:
+ //if(in_array($scriptVariable, ['', 'ytInitialData']))
return $json;
}so can leverage and complete unit tests of:
php test.php videos 2> /dev/nullIt is not a problem that some unit test fail both locally and on the official instance as it means that the associated feature does not seem to work anymore.
On my laptop:
php test.php videos 2> /dev/nullPASS videos part=id&clipId=UgkxU2HSeGL_NvmDJ-nQJrlLwllwMDBdGZFs items/0/videoId NiXD4xVJM5Y
PASS videos part=clip&clipId=UgkxU2HSeGL_NvmDJ-nQJrlLwllwMDBdGZFs items/0/clip Array
PASS videos part=contentDetails&id=g5xNzUA5Qf8 items/0/contentDetails/duration 213
PASS videos part=status&id=J8ZVxDK11Jo items/0/status/embeddable
PASS videos part=status&id=g5xNzUA5Qf8 items/0/status/embeddable 1
PASS videos part=short&id=NiXD4xVJM5Y items/0/short/available
PASS videos part=short&id=ydPkyvWtmg4 items/0/short/available 1
PASS videos part=musics&id=DUT5rEU6pqM items/0/musics/0 Array
PASS videos part=musics&id=4sC3mbkP_x8 items/0/musics Array
PASS videos part=music&id=FliCdfxdtTI items/0/music/available
PASS videos part=music&id=ntG3GQdY_Ok items/0/music/available 1
PASS videos part=isPaidPromotion&id=Q6gtj1ynstU items/0/isPaidPromotion
PASS videos part=isPaidPromotion&id=PEorJqo2Qaw items/0/isPaidPromotion 1
PASS videos part=isMemberOnly&id=Q6gtj1ynstU items/0/isMemberOnly
PASS videos part=isMemberOnly&id=Ln9yZDtfcWg items/0/isMemberOnly 1
PASS videos part=qualities&id=IkXH9H2ofa0 items/0/qualities Array
PASS videos part=chapters&id=n8vmXvoVjZw items/0/chapters Array
PASS videos part=isOriginal&id=FliCdfxdtTI items/0/isOriginal
FAIL videos part=isOriginal&id=iqKdEhx-dD4 items/0/isOriginal 1
PASS videos part=isPremium&id=FliCdfxdtTI items/0/isPremium
PASS videos part=isPremium&id=dNJMI92NZJ0 items/0/isPremium 1
PASS videos part=isRestricted&id=IkXH9H2ofa0 items/0/isRestricted
PASS videos part=isRestricted&id=ORdWE_ffirg items/0/isRestricted 1
PASS videos part=snippet&id=IkXH9H2ofa0 items/0/snippet Array
PASS videos part=activity&id=V6z0qF54RZ4 items/0/activity Array
PASS videos part=mostReplayed&id=XiCrniLQGYc items/0/mostReplayed/markers/0/intensityScoreNormalized 1
PASS videos part=explicitLyrics&id=Ehoe35hTbuY items/0/explicitLyrics
PASS videos part=explicitLyrics&id=PvM79DJ2PmM items/0/explicitLyrics 1So there is a single FAIL.
On the official instance:
php test.php videos 2> /dev/nullPASS videos part=id&clipId=UgkxU2HSeGL_NvmDJ-nQJrlLwllwMDBdGZFs items/0/videoId NiXD4xVJM5Y
PASS videos part=clip&clipId=UgkxU2HSeGL_NvmDJ-nQJrlLwllwMDBdGZFs items/0/clip Array
FAIL videos part=contentDetails&id=g5xNzUA5Qf8 items/0/contentDetails/duration 213 0
PASS videos part=status&id=J8ZVxDK11Jo items/0/status/embeddable
FAIL videos part=status&id=g5xNzUA5Qf8 items/0/status/embeddable 1
PASS videos part=short&id=NiXD4xVJM5Y items/0/short/available
PASS videos part=short&id=ydPkyvWtmg4 items/0/short/available 1
PASS videos part=musics&id=DUT5rEU6pqM items/0/musics/0 Array
PASS videos part=musics&id=4sC3mbkP_x8 items/0/musics Array
PASS videos part=music&id=FliCdfxdtTI items/0/music/available
PASS videos part=music&id=ntG3GQdY_Ok items/0/music/available 1
PASS videos part=isPaidPromotion&id=Q6gtj1ynstU items/0/isPaidPromotion
FAIL videos part=isPaidPromotion&id=PEorJqo2Qaw items/0/isPaidPromotion 1
PASS videos part=isMemberOnly&id=Q6gtj1ynstU items/0/isMemberOnly
PASS videos part=isMemberOnly&id=Ln9yZDtfcWg items/0/isMemberOnly 1
FAIL videos part=qualities&id=IkXH9H2ofa0 items/0/qualities Array Array
PASS videos part=chapters&id=n8vmXvoVjZw items/0/chapters Array
PASS videos part=isOriginal&id=FliCdfxdtTI items/0/isOriginal
FAIL videos part=isOriginal&id=iqKdEhx-dD4 items/0/isOriginal 1
PASS videos part=isPremium&id=FliCdfxdtTI items/0/isPremium
PASS videos part=isPremium&id=dNJMI92NZJ0 items/0/isPremium 1
PASS videos part=isRestricted&id=IkXH9H2ofa0 items/0/isRestricted
FAIL videos part=isRestricted&id=ORdWE_ffirg items/0/isRestricted 1
PASS videos part=snippet&id=IkXH9H2ofa0 items/0/snippet Array
PASS videos part=activity&id=V6z0qF54RZ4 items/0/activity Array
PASS videos part=mostReplayed&id=XiCrniLQGYc items/0/mostReplayed/markers/0/intensityScoreNormalized 1
PASS videos part=explicitLyrics&id=Ehoe35hTbuY items/0/explicitLyrics
PASS videos part=explicitLyrics&id=PvM79DJ2PmM items/0/explicitLyrics 1So there are 5 other FAILs including current issue.
minimizeCURL.py curl.sh paidContentOverlay...
curl 'https://www.youtube.com/watch?v=PEorJqo2Qaw' -H 'Cookie: __Secure-1PSIDTS=sidts-XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX; __Secure-1PSID=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX'Being precise to reduce requests leveraging authentication may help not make these credentials being detected as automated.
Concerning status and contentDetails parts, the official instance gets:
{
"responseContext": {
"visitorData": "CgtEQ0tkRnNLal9zQSic6tO2BjIiCgJGUhIcEhgSFgsMDg8QERITFBUWFxgZGhscHR4fICEgSg%3D%3D",
"serviceTrackingParams": [
{
"service": "GFEEDBACK",
"params": [
{
"key": "is_alc_surface",
"value": "false"
},
{
"key": "is_viewed_live",
"value": "False"
},
{
"key": "ipcc",
"value": "0"
},
{
"key": "logged_in",
"value": "0"
},
{
"key": "e",
"value": "23804281,23966208,23998056,24004644,24077241,24181174,24241378,24290971,24425063,24439361,24468724,24542367,24548629,24566687,51009781,51010235,51017346,51020570,51025415,51030103,51037342,51037353,51041512,51050361,51053689,51057844,51057851,51063643,51064835,51098297,51098299,51111738,51115184,51116067,51123611,51124104,51133103,51138234,51149607,51152050,51157411,51157841,51158514,51160545,51162170,51165467,51169118,51176511,51177012,51177817,51178314,51178327,51178346,51178355,51178982,51183910,51184022,51186528,51190057,51190073,51190082,51190085,51190200,51190209,51190220,51190229,51190652,51195231,51196181,51196478,51197687,51197694,51197701,51197706,51200253,51200260,51200293,51200300,51201350,51201365,51201372,51201381,51201426,51201433,51201440,51201447,51204329,51209050,51212458,51212466,51212546,51212555,51212569,51217504,51219800,51221011,51221150,51222972,51223962,51224135,51227037,51227291,51227410,51227776,51228350,51228352,51228767,51228778,51228783,51228796,51228809,51228812,51230241,51230389,51230478,51231373,51231814,51234407,51235080,51236017,51237842,51239093,51241028,51242269,51242447,51243940,51245822,51245831,51246283,51246305,51248255,51248734,51248739,51248747,51248748,51249769,51251508,51251675,51251811,51251836,51251849,51255677,51255681,51255743,51256074,51256084,51256732,51257533,51257852,51257902,51257918,51258066,51258360,51258457,51258835,51259133,51260592,51260634,51264983,51265341,51265356,51265377,51266743,51266946,51267567,51268387,51268978,51269632,51270086,51270830,51271256,51271390,51271669,51272491,51272506,51272513,51272530,51272570,51272589,51272663,51272710,51273423,51273446,51275152,51275157,51275172,51275185,51275194,51275206,51277508,51278034,51280249,51281600,51282073,51282084,51282508,51284157,51285417"
}
]
},
{
"service": "CSI",
"params": [
{
"key": "c",
"value": "WEB_EMBEDDED_PLAYER"
},
{
"key": "cver",
"value": "1.9999099"
},
{
"key": "yt_li",
"value": "0"
},
{
"key": "GetPlayer_rid",
"value": "0xa826c5e2625c823f"
}
]
},
{
"service": "GUIDED_HELP",
"params": [
{
"key": "logged_in",
"value": "0"
}
]
},
{
"service": "ECATCHER",
"params": [
{
"key": "client.version",
"value": "20240902"
},
{
"key": "client.name",
"value": "WEB_EMBEDDED_PLAYER"
}
]
}
],
"maxAgeSeconds": 0
},
"playabilityStatus": {
"status": "LOGIN_REQUIRED",
"reason": "Sign in to confirm you’re not a bot",
"errorScreen": {
"playerErrorMessageRenderer": {
"subreason": {
"runs": [
{
"text": "This helps protect our community. "
},
{
"text": "Learn more",
"navigationEndpoint": {
"clickTrackingParams": "CAAQu2kiEwjGjenI76KIAxXFZE8EHadtFNc=",
"urlEndpoint": {
"url": "https://support.google.com/youtube/answer/3037019#zippy=%2Ccheck-that-youre-signed-into-youtube"
}
}
}
]
},
"reason": {
"runs": [
{
"text": "Sign in to confirm you’re not a bot"
}
]
},
"proceedButton": {
"buttonRenderer": {
"trackingParams": "CAEQ8FsiEwjGjenI76KIAxXFZE8EHadtFNc="
}
},
"thumbnail": {
"thumbnails": [
{
"url": "//s.ytimg.com/yts/img/meh7-vflGevej7.png",
"width": 140,
"height": 100
}
]
},
"icon": {
"iconType": "ERROR_OUTLINE"
}
}
},
"contextParams": "Q0FFU0FnZ0M="
},
"trackingParams": "CAAQu2kiEwjGjenI76KIAxXFZE8EHadtFNc=",
"adBreakHeartbeatParams": "Q0FBJTNE"
}With:
diff --git a/common.php b/common.php
index 50fe0c2..c8a2d1e 100644
--- a/common.php
+++ b/common.php
@@ -19,10 +19,24 @@
}
}
- function getContextFromOpts($opts)
+ function implodeArray($anArray, $separator)
{
- if (GOOGLE_ABUSE_EXEMPTION !== '') {
- $cookieToAdd = 'GOOGLE_ABUSE_EXEMPTION=' . GOOGLE_ABUSE_EXEMPTION;
+ return array_map(fn($k, $v) => "${k}${separator}${v}", array_keys($anArray), array_values($anArray));
+ }
+
+ function getContextFromOpts($opts, $mayRequireAuthentication = false)
+ {
+ if (GOOGLE_ABUSE_EXEMPTION !== '' or $mayRequireAuthentication) {
+ $cookiesToAdd = [];
+ if (GOOGLE_ABUSE_EXEMPTION !== '') {
+ $cookiesToAdd['GOOGLE_ABUSE_EXEMPTION'] = GOOGLE_ABUSE_EXEMPTION;
+ }
+ if ($mayRequireAuthentication) {
+ // Both or equivalent `1` instead of `3` are needed for `videos?part=isPaidPromotion&id=PEorJqo2Qaw`.
+ $cookiesToAdd['__Secure-3PSID'] = 'XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX';
+ $cookiesToAdd['__Secure-3PSIDTS'] = 'sidts-XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX';
+ }
+ $cookiesToAdd = implode('; ', implodeArray($cookiesToAdd, '='));
// Can't we simplify the following code?
if (array_key_exists('http', $opts)) {
$http = $opts['http'];
@@ -31,20 +45,20 @@
$isThereACookieHeader = false;
foreach ($headers as $headerIndex => $header) {
if (str_starts_with($header, 'Cookie: ')) {
- $opts['http']['header'][$headerIndex] = "$header; $cookieToAdd";
+ $opts['http']['header'][$headerIndex] = "$header; $cookiesToAdd";
$isThereACookieHeader = true;
break;
}
}
if (!$isThereACookieHeader) {
- array_push($opts['http']['header'], "Cookie: $cookieToAdd");
+ array_push($opts['http']['header'], "Cookie: $cookiesToAdd");
}
}
} else {
$opts = [
'http' => [
'header' => [
- "Cookie: $cookieToAdd"
+ "Cookie: $cookiesToAdd"
]
]
];
@@ -61,7 +75,7 @@
return $headers;
}
- function fileGetContentsAndHeadersFromOpts($url, $opts)
+ function fileGetContentsAndHeadersFromOpts($url, $opts, $mayRequireAuthentication)
{
if(HTTPS_PROXY_ADDRESS !== '')
{
@@ -78,7 +92,7 @@
$opts['http']['header'] = $headers;
}
}
- $context = getContextFromOpts($opts);
+ $context = getContextFromOpts($opts, $mayRequireAuthentication);
$result = file_get_contents($url, false, $context);
return [$result, $http_response_header];
}
@@ -98,9 +112,9 @@
return $code == 303;
}
- function getRemote($url, $opts = [], $verifyTrafficIfForbidden = true)
+ function getRemote($url, $opts = [], $verifyTrafficIfForbidden = true, $mayRequireAuthentication = false)
{
- [$result, $headers] = fileGetContentsAndHeadersFromOpts($url, $opts);
+ [$result, $headers] = fileGetContentsAndHeadersFromOpts($url, $opts, $mayRequireAuthentication);
foreach (HTTP_CODES_DETECTED_AS_SENDING_UNUSUAL_TRAFFIC as $HTTP_CODE_DETECTED_AS_SENDING_UNUSUAL_TRAFFIC) {
if (str_contains($headers[0], strval($HTTP_CODE_DETECTED_AS_SENDING_UNUSUAL_TRAFFIC)) && ($HTTP_CODE_DETECTED_AS_SENDING_UNUSUAL_TRAFFIC != 403 || $verifyTrafficIfForbidden)) {
detectedAsSendingUnusualTraffic();
@@ -152,9 +166,9 @@
return getJSONStringFromHTMLScriptPrefix($html, "$prefix$scriptVariable = ");
}
- function getJSONFromHTML($url, $opts = [], $scriptVariable = '', $prefix = 'var ', $forceLanguage = false, $verifiesChannelRedirection = false)
+ function getJSONFromHTML($url, $opts = [], $scriptVariable = '', $prefix = 'var ', $forceLanguage = false, $verifiesChannelRedirection = false, $mayRequireAuthentication = false)
{
- $html = getRemote($url, $opts);
+ $html = getRemote($url, $opts, mayRequireAuthentication: $mayRequireAuthentication);
$jsonStr = getJSONStringFromHTML($html, $scriptVariable, $prefix);
$json = json_decode($jsonStr, true);
if($verifiesChannelRedirection)
diff --git a/videos.php b/videos.php
index b101d3c..41e7b78 100644
--- a/videos.php
+++ b/videos.php
@@ -98,9 +98,20 @@
'Content-Type: application/json',
'Accept-Language: en'
];
- if ($music) {
- array_push($headers, 'Referer: https://music.youtube.com');
- }
+ // As I am unable to reproduce a request to the specified URL for `!$music`, as the `$music` case, works let us use it for both ca
ses.
+ // Maybe can share `__Secure-3PSID` with `getContextFromOpts`.
+ $currentTime = time();
+ $SAPISID = 'XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX';
+ $__Secure_3PSID = 'XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX';
+ $ORIGIN = 'https://music.youtube.com';
+ $SAPISIDHASH = "${currentTime}_" . sha1("$currentTime $SAPISID $ORIGIN");
+
+ array_push($headers,
+ //'Referer: https://music.youtube.com',
+ "Origin: $ORIGIN",
+ "Authorization: SAPISIDHASH $SAPISIDHASH",
+ "Cookie: __Secure-3PSID=$__Secure_3PSID; __Secure-3PAPISID=$SAPISID",
+ );
$opts = [
'http' => [
'method' => 'POST',
@@ -266,7 +276,7 @@
}
if ($options['isPaidPromotion']) {
- $json = getJSONFromHTML("https://www.youtube.com/watch?v=$id", scriptVariable: 'ytInitialPlayerResponse');
+ $json = getJSONFromHTML("https://www.youtube.com/watch?v=$id", scriptVariable: 'ytInitialPlayerResponse', mayRequireAuthentication: true);
$isPaidPromotion = array_key_exists('paidContentOverlay', $json);
$item['isPaidPromotion'] = $isPaidPromotion;
}
@@ -326,7 +336,7 @@
}
if ($options['qualities']) {
- $json = getJSONFromHTML("https://www.youtube.com/watch?v=$id", scriptVariable: 'ytInitialPlayerResponse');
+ $json = getJSONFromHTML("https://www.youtube.com/watch?v=$id", scriptVariable: 'ytInitialPlayerResponse', mayRequireAuthentication: true);
$qualities = [];
foreach ($json['streamingData']['adaptiveFormats'] as $quality) {
if (array_key_exists('qualityLabel', $quality)) {
@@ -386,7 +396,7 @@
'header' => ['Cookie: PREF=f2=8000000'],
]
];
- $json = getJSONFromHTML("https://www.youtube.com/watch?v=$id", $opts, 'ytInitialPlayerResponse');
+ $json = getJSONFromHTML("https://www.youtube.com/watch?v=$id", $opts, 'ytInitialPlayerResponse', mayRequireAuthentication: true);
$playabilityStatus = $json['playabilityStatus'];
$isRestricted = array_key_exists('isBlockedInRestrictedMode', $playabilityStatus);
$item['isRestricted'] = $isRestricted;only get above local FAIL (after having closed Firefox private window). Used #258#issuecomment-2028886257.
Being notified when the official instance answers an unexpected value could help but these would need to be exhaustive which is not the case of https://yt0.lemnoslife.com/metrics/ so let us postpone this, as current part=mostReplayed check does not require authentication.
i300220
commented
1 week ago Interesting. I can confirm running a local instance works.
i300220
commented
1 week ago Tried from home, just for test...
$:~/web/YouTube-operational-API$ git pull
remote: Enumerating objects: 28, done.
remote: Counting objects: 100% (28/28), done.
remote: Compressing objects: 100% (10/10), done.
remote: Total 20 (delta 18), reused 12 (delta 10), pack-reused 0 (from 0)
Dépaquetage des objets: 100% (20/20), 3.13 Kio | 178.00 Kio/s, fait.
Depuis https://github.com/Benjamin-Loison/YouTube-operational-API
358770e..e4d1f24 main -> origin/main
Mise à jour 358770e..e4d1f24
Fast-forward
playlistItems.php | 9 ++++-----
tools/checkOperationnalAPI.py | 2 +-
tools/getJSONPathFromKey.py | 2 +-
tools/minimizeCURL.py | 2 +-
tools/simplifyCURL.py | 2 +-
videos.php | 27 +++++++++++++++++++--------
6 files changed, 27 insertions(+), 17 deletions(-)
$:~/web/YouTube-operational-API/ytPrivate$ php test.php videos 2> /dev/null
FAIL videos part=id&clipId=UgkxU2HSeGL_NvmDJ-nQJrlLwllwMDBdGZFs items/0/videoId NiXD4xVJM5Y
FAIL videos part=clip&clipId=UgkxU2HSeGL_NvmDJ-nQJrlLwllwMDBdGZFs items/0/clip
FAIL videos part=contentDetails&id=g5xNzUA5Qf8 items/0/contentDetails/duration 213
FAIL videos part=status&id=J8ZVxDK11Jo items/0/status/embeddable
FAIL videos part=status&id=g5xNzUA5Qf8 items/0/status/embeddable 1
FAIL videos part=short&id=NiXD4xVJM5Y items/0/short/available
FAIL videos part=short&id=ydPkyvWtmg4 items/0/short/available 1
FAIL videos part=musics&id=DUT5rEU6pqM items/0/musics/0
FAIL videos part=musics&id=4sC3mbkP_x8 items/0/musics
FAIL videos part=music&id=FliCdfxdtTI items/0/music/available
FAIL videos part=music&id=ntG3GQdY_Ok items/0/music/available 1
FAIL videos part=isPaidPromotion&id=Q6gtj1ynstU items/0/isPaidPromotion
FAIL videos part=isPaidPromotion&id=PEorJqo2Qaw items/0/isPaidPromotion 1
FAIL videos part=isMemberOnly&id=Q6gtj1ynstU items/0/isMemberOnly
FAIL videos part=isMemberOnly&id=Ln9yZDtfcWg items/0/isMemberOnly 1
FAIL videos part=qualities&id=IkXH9H2ofa0 items/0/qualities
FAIL videos part=chapters&id=n8vmXvoVjZw items/0/chapters
FAIL videos part=isOriginal&id=FliCdfxdtTI items/0/isOriginal
FAIL videos part=isOriginal&id=iqKdEhx-dD4 items/0/isOriginal 1
FAIL videos part=isPremium&id=FliCdfxdtTI items/0/isPremium
FAIL videos part=isPremium&id=dNJMI92NZJ0 items/0/isPremium 1
FAIL videos part=isRestricted&id=IkXH9H2ofa0 items/0/isRestricted
FAIL videos part=isRestricted&id=ORdWE_ffirg items/0/isRestricted 1
FAIL videos part=snippet&id=IkXH9H2ofa0 items/0/snippet
FAIL videos part=activity&id=V6z0qF54RZ4 items/0/activity
FAIL videos part=mostReplayed&id=XiCrniLQGYc items/0/mostReplayed/markers/0/intensityScoreNormalized 1
FAIL videos part=explicitLyrics&id=Ehoe35hTbuY items/0/explicitLyrics
FAIL videos part=explicitLyrics&id=PvM79DJ2PmM items/0/explicitLyrics 1There is no documentation for work in progress unit tests, so I was not expecting you to run them. If you remove 2> /dev/null you will probably notice what is going wrong.
Also note that some tests need actual JSON to match against, those JSON have not all yet been publicly published as I am looking for better ways to store these expectated JSONs.
i300220
commented
1 week ago OK no problem. I'm just curious. Got more from apache error log. For each request:
PHP Warning: file_get_contents(tests/part=clip&clipId=UgkxU2HSeGL_NvmDJ-nQJrlLwllwMDBdGZFs.json): Failed to open stream: No such file or directory in /var/www/sites/YouTube-operational-API/videos.php on line 7
PHP Stack trace:
PHP 1. {main}() /var/www/sites/YouTube-operational-API/ytPrivate/test.php:0
PHP 2. require_once() /var/www/sites/YouTube-operational-API/ytPrivate/test.php:27
PHP 3. file_get_contents($filename = 'tests/part=clip&clipId=UgkxU2HSeGL_NvmDJ-nQJrlLwllwMDBdGZFs.json') /var/www/sites/YouTube-operational-API/videos.php:7Was probably expecting tests.php.
Will add AllowOverride All for this dir.
Benjamin-Loison
commented
1 week ago I guess you meant tests/ instead of tests.php.
i300220
commented
1 week ago No, tests.php. The .htaccess was not processed.
Now that it is, it tries to execute php-cgi, and I use php-fpm.
sh: 1: php-cgi: not found
FAIL videos part=explicitLyrics&id=PvM79DJ2PmM items/0/explicitLyrics 1
:~/web/YouTube-operational-API/ytPrivate$ grep php-cgi *
test.php: $retrievedContent = shell_exec("php-cgi ../$endpoint.php " . escapeshellarg($url));You can try sudo apt install php-cgi.
i300220
commented
1 week ago Sure but not before a complete backup. Had terrible issues in the past on Debian. The 2 were conflicting.
Benjamin-Loison
commented
1 week ago If you are curious, then know that current tests/ are available at https://yt.lemnoslife.com/ytPrivate/tests/.
About the issue itself I logged in with an ad-hoc account on the official instance (see #issuecomment-2323470675) solving your issue:
curl 'https://yt.lemnoslife.com/videos?part=contentDetails&id=n8vmXvoVjZw'{
"kind": "youtube#videoListResponse",
"etag": "NotImplemented",
"items": [
{
"kind": "youtube#video",
"etag": "NotImplemented",
"id": "n8vmXvoVjZw",
"contentDetails": {
"duration": 1222
}
}
]
}thank you for having reported this issue.
i300220
commented
1 week ago Thanks a lot! You're very welcome.
Fix confirmed.
Benjamin-Loison
commented
1 week ago Note that the issue may come back in the future, do not hesitate to re-open this issue if it is the case. #issuecomment-2323470675 mentions the missing proper monitoring.
Benjamin-Loison
commented
1 week ago Related to YoutubeExplode/issues/794.
For a couple of weeks, impossible to get the duration of a YT video. It always returns 0. Perfectly repeatable.
e.g. https://yt.lemnoslife.com/videos?part=contentDetails&id=n8vmXvoVjZw