search

iejMac / video2dataset

Easily create large video dataset from video urls
MIT License
543 stars 65 forks source link

ERROR: [youtube] : Sign in to confirm you’re not a bot. #348

Open YifanXu74 opened 3 months ago

YifanXu74 commented 3 months ago

I recently tried downloading the dataset, but it comes out such error:

ERROR: [youtube] ---LyFa59Go: Sign in to confirm you’re not a bot. This helps protect our community

It seems that YouTube has updated some policies to block web crawlers using yt-dlp. I tried to address the problem using a solution here, but but soon hit the request limit and was blocked again, see:

[youtube+oauth2] -2WrOGUnUHM: You’ve hit the request limit.. Your limits will reset soon, usually within an hour.

This dataset is important for my research. I'd appreciate any clues or solutions on this problem!

markus583 commented 2 months ago

+1

jwu-sym commented 2 months ago

I signed in to confirm I am not a bot. It looked working. Steps :

  1. My yt-dlp was installed in a linux server box, no GUI.
  2. Install a docker with GUI on the same box, I used "docker run -p 6080:80 -v /dev/shm:/dev/shm dorowu/ubuntu-desktop-lxde-vnc"
  3. Visit http://docker-server:6080, spawn a firefox and sign in to YouTube
  4. Transfer .mozilla directory from docker root directory to yt-dlp box. that runs yt-dlp
  5. alias yt-dlp='yt-dlp --cookies-from-browser firefox' It runs properly. Your mile might differ. The idea is 'Sign in'.
SuhatAkbulak commented 2 weeks ago

+1