ytdl-org / youtube-dl

Command-line program to download videos from YouTube.com and other video sites

http://ytdl-org.github.io/youtube-dl/

The Unlicense

131.62k stars 9.97k forks source link

nitter support #27483

Open Soundtoxin opened 3 years ago

Soundtoxin commented 3 years ago

Checklist

[x] I'm reporting a new site support request
[x] I've verified that I'm running youtube-dl version 2020.12.14
[x] I've checked that all provided URLs are alive and playable in a browser
[x] I've checked that none of provided URLs violate any copyrights
[x] I've searched the bugtracker for similar site support requests including closed ones

Example URLs

Single video: https://nitter.net/DanielFore/status/1340023143990054913#m
Single video: https://nitter.net/nathandyer_/status/1340111249359646727
Single video: https://nitter.snopyta.org/ArtemisVtuber/status/1340029333763575808

Description

Sometimes a video will open, but it won't be the full length, or it will be a random video in the thread and not the one in the exact linked post

https://github.com/zedeus/nitter/wiki/Instances list of instances to support here

fossdd commented 3 years ago

It isn't very nice that there is to implement multiple instances like here:

https://github.com/ytdl-org/youtube-dl/blob/bbe8cc6662ce870b2efee5d2a3fceab39e1a3c2c/youtube_dl/extractor/peertube.py#L20-L417

EdenOvrutskiy commented 3 years ago

I tried doing this, and for some reason I get two different versions of the page's source. https://www.diffchecker.com/U1Di2koN chrome version on the left, yt-dl's version on the right. The m3u8 file on the site is generated by the form on the yt-dl source with a request somehow...

Never encountered a situation where "view-source" has unique information based on where I view it from. And, I see no javascript to send that hidden form anywhere..

Any tips on how to get the m3u8's link on the module?

selfisekai commented 3 years ago

26990

B0pol commented 3 years ago

@SpiderRider067 some instances enable by default hls playback, nitter.net and the default config file do not. You have to add a cookie "hlsPlayback" to get video url link everytime, no matter the instance used. But I already worked on an extractor #26990 , I will rebase, update instance list soon and I hope it will get merged.

Mennaruuk commented 3 years ago

I know Nitter has been suffering from a rate-limiting problem. Is there a way to have youtube-dl retry if instance is rate-limited and the .m3u8 can't be extracted? Generally, a couple of refreshes solves the rate-limiting problem.