search

openzim / python-scraperlib

Collection of Python code to re-use across Python-based scrapers
GNU General Public License v3.0
18 stars 16 forks source link

YouTube downloader integration in scarperlib #29

Closed satyamtg closed 3 years ago

satyamtg commented 4 years ago

There are multiple scrapers for which need to download youtube videos using youtube_dl. However, this has a lot of complexities to deal with and a centralized youtube downloaded (which could handle the multiprocessing), can be implemented here in scraperlib. At this point of time, I don't have a very concrete idea on how shall this be done and whether we shall manage 429s, but I think this is something that would be a good to have feature.

rgaudin commented 4 years ago

Agrees on the need and the difficulty. Let's use this to list requirements for that feature.