snap-research / Panda-70M

[CVPR 2024] Panda-70M: Captioning 70M Videos with Multiple Cross-Modality Teachers
https://snap-research.github.io/Panda-70M/
528 stars 19 forks source link

ERROR: [youtube] : Sign in to confirm you’re not a bot. This helps protect our community #60

Open YifanXu74 opened 4 months ago

YifanXu74 commented 4 months ago

Hi,

I recently tried downloading the dataset, but it comes out such error:

ERROR: [youtube] ---LyFa59Go: Sign in to confirm you’re not a bot. This helps protect our community

It seems that YouTube has updated some policies to block web crawlers using yt-dlp. I tried to address the problem using a solution here, but but soon hit the request limit and was blocked again, see:

[youtube+oauth2] -2WrOGUnUHM: You’ve hit the request limit.. Your limits will reset soon, usually within an hour.

This dataset is important for my research. I'd appreciate any clues or solutions on this problem!

sun-yanjun commented 3 months ago

Hi, have you solved this problem? I’m facing the same issue.

Chuge0335 commented 3 months ago

Same error, is there any better solutions?

DmytroUsenko commented 3 months ago

I'll subscribe to this error as well. I got the same thing using youtube-dl, though.

Abdurrafey-Siddiqui commented 3 months ago

Same error, subscribed

taiyouZhang commented 3 months ago

same

GizuMoca commented 3 months ago

Has anyone found a solution? How many accesses will cause the limit?

Chuge0335 commented 3 months ago

Same error

weihxx commented 3 months ago

Same error

Hzzone commented 3 months ago

This issue usually arises when using yb-dlp to download YouTube videos, which I also experienced. You can resolve it by exporting cookies from your web browser to facilitate video downloads. However, be aware that YouTube may ban your account.

DmytroUsenko commented 3 months ago

This issue usually arises when using yb-dlp to download YouTube videos, which I also experienced. You can resolve it by exporting cookies from your web browser to facilitate video downloads. However, be aware that YouTube may ban your account.

could you share more details please how did you pass cookies?

Hzzone commented 3 months ago

I used the following shell command to download videos:

command = f'yt-dlp -o {video_path} --cookies {cookies_path} -f \"{format_string}\" {url}'

I export YouTube cookies using this extension: https://chromewebstore.google.com/detail/cclelndahbckbenkjhflpdbgdldlbecc. However, cookies expire after a while, so I need a way to automatically obtain new ones.

DmytroUsenko commented 3 months ago

unfortunately this does not work for me (

[youtube] Extracting URL: https://www.youtube.com/watch?v=UAna0bJUhhw
[youtube] UAna0bJUhhw: Downloading webpage
[debug] [youtube] Extracted SAPISID cookie
[debug] [youtube] Copying __Secure-3PAPISID cookie to SAPISID cookie
[youtube] UAna0bJUhhw: Downloading ios player API JSON
[youtube] UAna0bJUhhw: Downloading web creator player API JSON
ERROR: [youtube] UAna0bJUhhw: Sign in to confirm you’re not a bot. This helps protect our community. Learn more
DmytroUsenko commented 3 months ago

UPD. Fixed this issue today with this: https://github.com/coletdjnz/yt-dlp-youtube-oauth2

fangqi-Zhu commented 2 months ago

Is it possible to integrate this process into the download tutorial for this repository?

pikeyang commented 2 months ago

UPD. Fixed this issue today with this: https://github.com/coletdjnz/yt-dlp-youtube-oauth2

Can you explain in detail how it was done

deconasser commented 2 months ago

Do you have another solution for this issues ? I found troublesome with export cookies!

LuigiNixy commented 2 months ago

The videos can be downloaded with pytubefix.

jingwangsg commented 2 weeks ago

@LuigiNixy Hi, can you still use pytubefix to massively download videos now?