Benjamin-Loison / YouTube-operational-API

YouTube operational API works when YouTube Data API v3 fails.
401 stars 52 forks source link

`nextPageToken` doesn't work after one time works. #263

Closed quroom closed 7 months ago

quroom commented 7 months ago

When I test this code,

import requests

channelId = 'UCLbOYoltlCD0LN06fSwKTsQ'

baseUrl = f'https://yt.lemnoslife.com/channels' # ?part=shorts&id={channelId}

nextPageToken = ''
index = 1
while True:
    params = {
        'part': 'shorts',
        'id': channelId,
    }
    if nextPageToken:
        params['pageToken'] = nextPageToken
    url = baseUrl
    #url = baseUrl + (f'&pageToken={nextPageToken}' if nextPageToken else '')
    #params = None
    response = requests.get(url, params = params).json()
    item = response['items'][0]
    print(f"page{index}.============================")
    for video in item['shorts']:
        print(video['title'])
    nextPageToken = item.get('nextPageToken')
    if not nextPageToken:
        break
    index+=1

It works well until page 2. But I get this error after page 2. What is the problem?

{
    "error": {
        "code": 400,
        "message": "Invalid pageToken"
    }
}
Benjamin-Loison commented 7 months ago

Thank you for reporting.

Related to #264.

Personal notes:

URL='http://localhost/YouTube-operational-API/channels?part=shorts&id=UCLbOYoltlCD0LN06fSwKTsQ' && DATA=`curl -s "$URL"` && echo $DATA | jq .items[0].shorts[].videoId | wc -l && NEXT_PAGE_TOKEN=`echo $DATA | jq -r .items[0].nextPageToken` && curl "$URL&pageToken=$NEXT_PAGE_TOKEN"
import requests

CHANNEL_ID = 'UCLbOYoltlCD0LN06fSwKTsQ'

URL = 'http://localhost/YouTube-operational-API/channels'

params = {
    'part': 'shorts',
    'id': CHANNEL_ID,
}

index = 1
videoIds = set()

while True:
    item = requests.get(URL, params = params).json()['items'][0]
    print(f"page {index}")
    for video in item['shorts']:
        videoIds.add(video['videoId'])
    print(len(videoIds))
    nextPageToken = item.get('nextPageToken')
    if not nextPageToken:
        break
    index += 1
    params['pageToken'] = nextPageToken
    #print(nextPageToken)
quroom commented 7 months ago

Thank you for reporting.

Related to #264.

Personal notes:

URL='http://localhost/YouTube-operational-API/channels?part=shorts&id=UCLbOYoltlCD0LN06fSwKTsQ' && DATA=`curl -s "$URL"` && echo $DATA | jq .items[0].shorts[].videoId | wc -l && NEXT_PAGE_TOKEN=`echo $DATA | jq -r .items[0].nextPageToken` && curl "$URL&pageToken=$NEXT_PAGE_TOKEN"
import requests

CHANNEL_ID = 'UCLbOYoltlCD0LN06fSwKTsQ'

URL = 'http://localhost/YouTube-operational-API/channels'

params = {
    'part': 'shorts',
    'id': CHANNEL_ID,
}

index = 1
videoIds = set()

while True:
    item = requests.get(URL, params = params).json()['items'][0]
    print(f"page {index}")
    for video in item['shorts']:
        videoIds.add(video['videoId'])
    print(len(videoIds))
    nextPageToken = item.get('nextPageToken')
    if not nextPageToken:
        break
    index += 1
    params['pageToken'] = nextPageToken
    #print(nextPageToken)

Does this code works only in local host? Or does this code has issue?

Benjamin-Loison commented 7 months ago

It works both on localhost and with the official YouTube operational API instance. See YouTube-operational-API/wiki/Glossary/0be0be6df7142cd10f8c0ce1504b1c0a82dbb416 for more details.

If you want it to use the official YouTube operational API instance, then change:

5c5
< URL = 'http://localhost/YouTube-operational-API/channels'
---
> URL = 'https://yt.lemnoslife.com/channels'
quroom commented 7 months ago

It works both on localhost and with the official YouTube operational API instance. See YouTube-operational-API/wiki/Glossary/0be0be6df7142cd10f8c0ce1504b1c0a82dbb416 for more details.

If you want it to use the official YouTube operational API instance, then change:

5c5
< URL = 'http://localhost/YouTube-operational-API/channels'
---
> URL = 'https://yt.lemnoslife.com/channels'

It's weird situation. Same code didn't work yesterday. Now it works very well.

Benjamin-Loison commented 7 months ago

Benjamin-Loison closed this as completed in 4c01f0d 13 hours ago

Source: above

You are relying on https://yt.lemnoslife.com website which I modified the source code according to above commit to solve the issue you reported, so it is normal that your end-user code works now without any change, it was a YouTube operational API instance server-side issue that I solved.

quroom commented 7 months ago

@Benjamin-Loison Now I understand what you are saying. Thanks for quick response.