Benjamin-Loison / YouTube-operational-API

YouTube operational API works when YouTube Data API v3 fails.
384 stars 47 forks source link

The official instance is not operating anymore, as YouTube legal team requested me to do so #317

Open Benjamin-Loison opened 3 days ago

Benjamin-Loison commented 3 days ago

See https://yt.lemnoslife.com.

Benjamin-Loison commented 3 days ago

Could consider to some extent closing some no-key service related issues.

Benjamin-Loison commented 3 days ago

We can keep no-key service related stuff as it can help rotate between regular project one and one with increase quota even if it is less useless which is maybe authorized by YouTube. Otherwise it can be helpful for people having multiple keys from different regular projects but this may not be authorized, see https://discord.com/channels/933841502155706418/933841503103627316/1231169992167526440 or even leaked ones.

Initially thought about doing these modifications: ```diff diff --git a/addKey.php b/addKey.php deleted file mode 100644 index 1bf6d46..0000000 --- a/addKey.php +++ /dev/null @@ -1,42 +0,0 @@ - [ - 'ignore_errors' => true, - ] - ]; - $content = getJSON("https://www.googleapis.com/youtube/v3/videos?part=snippet&id=mWdFMNQBcjs&key=$key", $httpOptions, false); - // The force secret is used to store the YouTube Data API v3 even if it's not having quota, as we assume that the trusted instance that send it to this one has checked that it has quota. - if ($content['items'][0]['snippet']['title'] === 'A public video' || (isset($_GET['forceSecret']) && $_GET['forceSecret'] === ADD_KEY_FORCE_SECRET)) { - file_put_contents(KEYS_FILE, ($keysContent === '' || $keysContent === false ? '' : "\n") . $key, FILE_APPEND); - // Avoid sending another time the given key to all instances. - if (!isset($_GET['forceSecret'])) { - foreach (ADD_KEY_TO_INSTANCES as $addKeyToInstance) { - getRemote($addKeyToInstance . "addKey.php?key=$key&forceSecret=" . ADD_KEY_FORCE_SECRET); - } - } - echo 'YouTube Data API v3 key added.'; - } elseif ($content['error']['errors'][0]['reason'] === 'quotaExceeded') { - // As users can set `Queries per minute` quota to 0, we avoid denial-of-service by not considering them. - echo 'Not adding YouTube Data API v3 key having quota exceeded.'; - } else { - // This YouTube Data API API v3 keys isn't assigned or there is another error. - echo 'Incorrect YouTube Data API v3 key.'; - } - } else { - echo 'This YouTube Data API v3 key is already in the list.'; - } - } else { - echo "The key provided isn't a YouTube Data API v3 key."; - } - } diff --git a/configuration.php b/configuration.php index 58ae206..4d2db31 100644 --- a/configuration.php +++ b/configuration.php @@ -12,12 +12,4 @@ define('HTTPS_PROXY_USERNAME', ''); define('HTTPS_PROXY_PASSWORD', ''); - // No-key endpoint: - define('KEYS_FILE', 'ytPrivate/keys.txt'); - // Both following entries can be generated with `tr -dc A-Za-z0-9 diff --git a/index.php b/index.php index b8e4e02..591aed6 100644 --- a/index.php +++ b/index.php @@ -248,38 +248,6 @@ ?> -

Make request WITHOUT ANY KEY:

- -

To make ANY request WITHOUT ANY KEY/USING YOUR QUOTA, you can use:

-

For instance you can use: instead of

-

I may add in the future limitation per IP etc if the quota need to be better shared among the persons using this API.

- -

Currently this service is powered by keys.

- - - -Share your YouTube Data API v3 key to power the no-key service: > -

Open-source:

The source code is available on GitHub: ```

As https://discord.com/channels/933841502155706418/933841503103627316/1294448943739961514 mentions using multiple instances.

Benjamin-Loison commented 3 days ago

If everyone was fine with that removing https://yt.lemnoslife.com for just keeping the repository seems to make sense. Well .htaccess is still necessary for redirections. So instead of removing if everyone was fine with that could set a homepage redirecting the repository etc like https://ytdl-org.github.io/youtube-dl/.

curl -I https://yt.lemnoslife.com/matrix

helps.

I may would like to have a clever documentation generation to avoid committing with git an expanded algorithm result. Maybe GitHub page is an interesting alternative.

Benjamin-Loison commented 3 days ago

Should automate Oracle account creation with Online authentication API, then automate instance creation and setting up YouTube operational API instance.

IPv4=XXX.XXX.XXX.XXX
ssh ubuntu@$IPv4
sudo apt update && sudo apt upgrade -y && sudo apt autoremove -y && sudo apt dist-upgrade -y

sudo apt install -y apache2 php git
sudo a2enmod rewrite headers

# Replace `AllowOverride None` with `AllowOverride All` in `<Directory /var/www/>` in `/etc/apache2/apache2.conf`.

# Is not reload, enough?
sudo service apache2 restart

sudo chown -R www-data: /var/www/

sudo apt install -y composer protobuf-compiler

sudo su -l www-data -s /bin/bash
cd html/
rm index.html

git clone https://github.com/Benjamin-Loison/YouTube-operational-API .

composer require google/protobuf

protoc --php_out=proto/php/ --proto_path=proto/prototypes/ $(find proto/prototypes/ -type f)

The Server Fault answer 200658 or/and https://cloud.oracle.com/networking/vcns/ocid1.vcn.oc1.eu-marseille-1.XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/security-lists/ocid1.securitylist.oc1.eu-marseille-1.XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX?region=eu-marseille-1 are necessary to make curl http://XXX.XXX.XXX.XXX work fine.

Related to Benjamin_Loison/certbot/issues/4.

curl -s http://$IPv4 | grep 'This instance'
curl -s http://$IPv4/channels?handle=@MrBeast | jq .items[0].id
<br/><br/>This instance (my instance) uses version: <a href="https://github.com/Benjamin-Loison/YouTube-operational-API/commit/7e5798628d8a547199b18d91d9bbfd7006d16866
"UCX6OQ3DkcsbYNE6H8uQQuVA"
Benjamin-Loison commented 3 days ago
curl -s https://www.youtube.com | grep -Po 'AIzaSy[A-D][a-zA-Z0-9-_]{32}' | uniq
Output: ``` AIzaSyAO_FJ2SlqU8Q4STEHLGCilw_Y9_11qcW8 AIzaSyDZNkyC-AtROwMBpLfevIvqYk-Gfi8ZOeo AIzaSyDophAQuyyiBr8h0nypEwXUKozH-BEswD0 AIzaSyBU2xE_JHvB6wag3tMfhxXpg2Q_W8xnM-I AIzaSyDVDUqts7CooOWu_Yyc_8s4f8Ywc-Oj9H4 ```
for line in `curl -s https://www.youtube.com | grep -Po 'AIzaSy[A-D][a-zA-Z0-9-_]{32}' | uniq`
do
    #echo "|$line|"
    echo $line
    curl "https://www.googleapis.com/youtube/v3/search?q=test&key=$line"
done
Output: ``` AIzaSyAO_FJ2SlqU8Q4STEHLGCilw_Y9_11qcW8 { "error": { "code": 403, "message": "YouTube Data API v3 has not been used in project 144809266352 before or it is disabled. Enable it by visiting https://console.developers.google.com/apis/api/youtube.googleapis.com/overview?project=144809266352 then retry. If you enabled this API recently, wait a few minutes for the action to propagate to our systems and retry.", "errors": [ { "message": "YouTube Data API v3 has not been used in project 144809266352 before or it is disabled. Enable it by visiting https://console.developers.google.com/apis/api/youtube.googleapis.com/overview?project=144809266352 then retry. If you enabled this API recently, wait a few minutes for the action to propagate to our systems and retry.", "domain": "usageLimits", "reason": "accessNotConfigured", "extendedHelp": "https://console.developers.google.com" } ], "status": "PERMISSION_DENIED", "details": [ { "@type": "type.googleapis.com/google.rpc.Help", "links": [ { "description": "Google developers console API activation", "url": "https://console.developers.google.com/apis/api/youtube.googleapis.com/overview?project=144809266352" } ] }, { "@type": "type.googleapis.com/google.rpc.ErrorInfo", "reason": "SERVICE_DISABLED", "domain": "googleapis.com", "metadata": { "service": "youtube.googleapis.com", "consumer": "projects/144809266352" } } ] } } AIzaSyDZNkyC-AtROwMBpLfevIvqYk-Gfi8ZOeo { "error": { "code": 403, "message": "Requests from referer \u003cempty\u003e are blocked.", "errors": [ { "message": "Requests from referer \u003cempty\u003e are blocked.", "domain": "global", "reason": "forbidden" } ], "status": "PERMISSION_DENIED", "details": [ { "@type": "type.googleapis.com/google.rpc.ErrorInfo", "reason": "API_KEY_HTTP_REFERRER_BLOCKED", "domain": "googleapis.com", "metadata": { "service": "youtube.googleapis.com", "consumer": "projects/929106728511" } } ] } } AIzaSyDophAQuyyiBr8h0nypEwXUKozH-BEswD0 { "error": { "code": 403, "message": "YouTube Data API v3 has not been used in project 512726152798 before or it is disabled. Enable it by visiting https://console.developers.google.com/apis/api/youtube.googleapis.com/overview?project=512726152798 then retry. If you enabled this API recently, wait a few minutes for the action to propagate to our systems and retry.", "errors": [ { "message": "YouTube Data API v3 has not been used in project 512726152798 before or it is disabled. Enable it by visiting https://console.developers.google.com/apis/api/youtube.googleapis.com/overview?project=512726152798 then retry. If you enabled this API recently, wait a few minutes for the action to propagate to our systems and retry.", "domain": "usageLimits", "reason": "accessNotConfigured", "extendedHelp": "https://console.developers.google.com" } ], "status": "PERMISSION_DENIED", "details": [ { "@type": "type.googleapis.com/google.rpc.Help", "links": [ { "description": "Google developers console API activation", "url": "https://console.developers.google.com/apis/api/youtube.googleapis.com/overview?project=512726152798" } ] }, { "@type": "type.googleapis.com/google.rpc.ErrorInfo", "reason": "SERVICE_DISABLED", "domain": "googleapis.com", "metadata": { "consumer": "projects/512726152798", "service": "youtube.googleapis.com" } } ] } } AIzaSyBU2xE_JHvB6wag3tMfhxXpg2Q_W8xnM-I { "error": { "code": 403, "message": "YouTube Data API v3 has not been used in project 558586699847 before or it is disabled. Enable it by visiting https://console.developers.google.com/apis/api/youtube.googleapis.com/overview?project=558586699847 then retry. If you enabled this API recently, wait a few minutes for the action to propagate to our systems and retry.", "errors": [ { "message": "YouTube Data API v3 has not been used in project 558586699847 before or it is disabled. Enable it by visiting https://console.developers.google.com/apis/api/youtube.googleapis.com/overview?project=558586699847 then retry. If you enabled this API recently, wait a few minutes for the action to propagate to our systems and retry.", "domain": "usageLimits", "reason": "accessNotConfigured", "extendedHelp": "https://console.developers.google.com" } ], "status": "PERMISSION_DENIED", "details": [ { "@type": "type.googleapis.com/google.rpc.Help", "links": [ { "description": "Google developers console API activation", "url": "https://console.developers.google.com/apis/api/youtube.googleapis.com/overview?project=558586699847" } ] }, { "@type": "type.googleapis.com/google.rpc.ErrorInfo", "reason": "SERVICE_DISABLED", "domain": "googleapis.com", "metadata": { "service": "youtube.googleapis.com", "consumer": "projects/558586699847" } } ] } } AIzaSyDVDUqts7CooOWu_Yyc_8s4f8Ywc-Oj9H4 { "error": { "code": 403, "message": "YouTube Data API v3 has not been used in project 209141348333 before or it is disabled. Enable it by visiting https://console.developers.google.com/apis/api/youtube.googleapis.com/overview?project=209141348333 then retry. If you enabled this API recently, wait a few minutes for the action to propagate to our systems and retry.", "errors": [ { "message": "YouTube Data API v3 has not been used in project 209141348333 before or it is disabled. Enable it by visiting https://console.developers.google.com/apis/api/youtube.googleapis.com/overview?project=209141348333 then retry. If you enabled this API recently, wait a few minutes for the action to propagate to our systems and retry.", "domain": "usageLimits", "reason": "accessNotConfigured", "extendedHelp": "https://console.developers.google.com" } ], "status": "PERMISSION_DENIED", "details": [ { "@type": "type.googleapis.com/google.rpc.Help", "links": [ { "description": "Google developers console API activation", "url": "https://console.developers.google.com/apis/api/youtube.googleapis.com/overview?project=209141348333" } ] }, { "@type": "type.googleapis.com/google.rpc.ErrorInfo", "reason": "SERVICE_DISABLED", "domain": "googleapis.com", "metadata": { "service": "youtube.googleapis.com", "consumer": "projects/209141348333" } } ] } } ```
Benjamin-Loison commented 3 days ago

So maybe with the right referrer AIzaSyDZNkyC-AtROwMBpLfevIvqYk-Gfi8ZOeo would be interesting.

DuckDuckGo and Google search "AIzaSyDZNkyC-AtROwMBpLfevIvqYk-Gfi8ZOeo" does not help.

node-ytpl/blob/ab62ca013de11b7a36ab159802c6fae15f00a477/test/pages/privatepage.html#L18 and similar results do not seem relevant.

https://www.joesandbox.com/analysis/1351525/1/html may be interesting.

Benjamin-Loison commented 3 days ago

Wikipedia: Cease and desist

Source: https://discord.com/channels/933841502155706418/933841503103627316/1294427765432647701

Benjamin-Loison commented 3 days ago

Should consider backing up mentioned in https://discord.com/channels/933841502155706418/933841503103627316/1283796566632562760.

Benjamin-Loison commented 3 days ago

Should consider potential unlimited quota, see https://discord.com/channels/933841502155706418/933841503103627316/1294441540156002315.

Related to #18.

Benjamin-Loison commented 3 days ago

Investigate https://www.futo.org notably https://chat.futo.org (source: https://discord.com/channels/933841502155706418/933841503103627316/1294426983962513570).

Benjamin-Loison commented 2 days ago

#11#issuecomment-1325539722 specifies what account maybe to not use to act for the moment at least as a regular user.

Let us delete redundant projects on my main Google account using YouTube Data API v3, as it may be against ToS and 10,000 should be enough quota for my personal usage.

Related to Benjamin_Loison/youtube_push_notifications/issues/13.