ytdl-org / youtube-dl

Command-line program to download videos from YouTube.com and other video sites
http://ytdl-org.github.io/youtube-dl/
The Unlicense
131.39k stars 9.96k forks source link

--prefer-free-formats does not prefer webm to mp4 #6018

Open kparal opened 9 years ago

kparal commented 9 years ago
$ youtube-dl 'https://www.youtube.com/watch?v=r9rGX91rq5I' -v -F
[debug] System config: [u'--prefer-free-formats']
[debug] User config: []
[debug] Command-line args: [u'https://www.youtube.com/watch?v=r9rGX91rq5I', u'-v', u'-F']
[debug] Encodings: locale UTF-8, fs UTF-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2015.06.04.1
[debug] Python version 2.7.10 - Linux-4.0.5-300.1.kparal.fc22.x86_64-x86_64-with-fedora-22-Twenty_Two
[debug] exe versions: ffmpeg 2.6.3, ffprobe 2.6.3
[debug] Proxy map: {}
[youtube] r9rGX91rq5I: Downloading webpage
[youtube] r9rGX91rq5I: Extracting video information
[youtube] r9rGX91rq5I: Downloading DASH manifest
[info] Available formats for r9rGX91rq5I:
format code  extension  resolution note
171          webm       audio only DASH audio  108k , audio@128k (44100Hz), 3.84MiB
140          m4a        audio only DASH audio  127k , m4a_dash container, aac  @128k (44100Hz), 4.76MiB
278          webm       256x144    DASH video   62k , webm container, VP9, 15fps, video only, 1.23MiB
242          webm       426x240    DASH video  112k , 30fps, video only, 1.88MiB
160          mp4        256x144    DASH video  122k , 15fps, video only, 3.37MiB
243          webm       640x360    DASH video  205k , 30fps, video only, 3.59MiB
244          webm       854x480    DASH video  281k , 30fps, video only, 6.15MiB
134          mp4        640x360    DASH video  297k , 30fps, video only, 3.44MiB
133          mp4        426x240    DASH video  307k , 30fps, video only, 7.54MiB
135          mp4        854x480    DASH video  626k , 30fps, video only, 7.05MiB
247          webm       1280x720   DASH video  645k , 30fps, video only, 14.85MiB
248          webm       1920x1080  DASH video 1204k , 30fps, video only, 27.42MiB
136          mp4        1280x720   DASH video 1286k , 30fps, video only, 14.79MiB
137          mp4        1920x1080  DASH video 2521k , 30fps, video only, 29.69MiB
271          webm       2560x1440  DASH video 3741k , 30fps, video only, 66.79MiB
264          mp4        2560x1440  DASH video 5990k , 30fps, video only, 96.12MiB
313          webm       3840x2160  DASH video 9743k , VP9, 30fps, video only, 157.88MiB
266          mp4        3840x2160  DASH video 10775k , h264, 30fps, video only, 152.25MiB
17           3gp        176x144    
36           3gp        320x240    
5            flv        400x240    
18           mp4        640x360    
43           webm       640x360    
22           mp4        1280x720   (best)

$ youtube-dl 'https://www.youtube.com/watch?v=r9rGX91rq5I' -v --get-format
[debug] System config: [u'--prefer-free-formats']
[debug] User config: []
[debug] Command-line args: [u'https://www.youtube.com/watch?v=r9rGX91rq5I', u'-v', u'--get-format']
[debug] Encodings: locale UTF-8, fs UTF-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2015.06.04.1
[debug] Python version 2.7.10 - Linux-4.0.5-300.1.kparal.fc22.x86_64-x86_64-with-fedora-22-Twenty_Two
[debug] exe versions: ffmpeg 2.6.3, ffprobe 2.6.3
[debug] Proxy map: {}
266 - 3840x2160 (DASH video)+140 - audio only (DASH audio)

Please note that [debug] System config: [u'--prefer-free-formats'] shows that I have preference to free formats enabled, but youtube-dl wants to download mp4 instead (266+140).

Hrxn commented 7 years ago

Well it doesn't always return file-size.

Example?

fireattack commented 7 years ago

AFAIK, all the DASH ones don't return file-size. Edit: not all DASH videos, but a lot of them are.

youtube-dl -F https://www.youtube.com/watch?v=eYztYCbV_8U

Which is also an example I mentioned before that reports wrong bitrate (original report: #11451), but it seems YouTube already corrected the bitrate issue for this particular one (format 136 now correctly shows lower bitrate than 137).

youtube-dl -F https://www.youtube.com/watch?v=iMhwshjSauo

This still shows no file-size information and wrong bitrate info

It reports

135          mp4        720x480    DASH video  198k , avc1.4d401e, 30fps, video only
133          mp4        360x240    DASH video  242k , avc1.4d400c, 30fps, video only

actual bit rates: 135: 112kbps 133: 32kbps


I know those are not "free formats", but this topic has already became the place to discuss "should we choose best video/audio merely based on bitrate (current behavior)", and the fact YouTube sometimes returns totally erroneous bit rate info should be taken into consideration before we start theorizing things like "weighted bitrate". I'm still baffled why we don't use resolution first as quality indicator, which has virtually no drawbacks.

cloph commented 7 years ago

To make everyone happy, perhaps we could keep --prefer-free-formats as-is (prefer highest resolution/quality and then free format if available)

@nyuszika7h : the problem is that this is not how it behaves as-is. If that was the case, everyone would be happy indeed. --prefer-free-formats as it is now acts like a --force-free-formats and this causes all the problem. (aka the need for overly long selector/filter lines instead of a simple [height <= 1080p] and --prefer-free-formats) see my previous comments for more details.

glenn-slayden commented 7 years ago

I agree with the comments made by @fireattack above here and here that resolution should be used instead of bitrate for the -f bestvideo determination.

Rather than continue to hijack the --prefer-free- debate, I created #14143 (albeit before discovering the comments I cited here in this unrelated issue).

One unqualified demerit of using bitrate--or for that matter, anything-rate--during the bestvideo determination is that the time denominator is irrelevant when comparing alike-length alternatives. Since the length is exactly the same for all the entries in a given youtube-dl -F format listing, you can multiply by every rate value in the table by the same duration (video length) to "simplify," or mathematically cancel the division which created that "rate."

Given a "most pristine" master copy which must exist for every video, entropy can be dissipated in several dimensions, but duration time is not one of them. Either the compression algorithms are more/less efficient, or the frame dimensions or frame rate are different. And it seems to me that given the special constraint discussed above, the latter two will swamp the former in all cases, making some sensible combination of dimension and fps a much more stable indicator of "best."

I think that, possibly, people have gotten in the mindset of elevating bitrate because it is quite likely a better indicator of quality when comparing dissimilar source material, but this habit may be obscuring better judgment when it comes to the specialized task of comparing all-alike material.

Irrefutably, really, bitrate is just a needlessly complex proxy for the total entropy each available format ultimately makes available. Add in the fact that the YouTube reported bitrate values are often wildly wrong (in fact far more so than this thread has reported; see #14143) and it's clear that by the nature of this type of selection task, bitrate is not the way to go.

So as I've noted, the -f bestvideo determination task, as with tasks in general, can and should capitalize on the special properties that are particular to it. In this case, this means recognizing the exclusion of time-denominated metrics as mathematically vacuous, and therefore unnecessarily complicating:, i.e., at best a harmless obfuscation, and at worst, exposing a tangibly increased error surface.

aufkrawall commented 6 years ago

I use this setting in mpv to always get VP9 + Opus/Vorbis, as suggested by utack (thanks again!): ytdl-format="((bestvideo[vcodec^=vp9]/bestvideo)+(bestaudio[acodec=opus]/bestaudio[acodec=vorbis]/bestaudio[acodec=aac]/bestaudio))/best" Is it possible to additionally also always prefer HDR video when it is available?

haasn commented 6 years ago

@aufkrawall Yeah, it gets identifies as vp9.2

I use this:

ytdl-format=(bestvideo[vcodec=vp9.2]/bestvideo[vcodec=vp9][fps>30]/bestvideo[vcodec=vp9][height>=1080]/bestvideo[fps>30]/bestvideo[height>720])+(bestaudio[acodec=opus]/bestaudio)/best
aufkrawall commented 6 years ago

Great, just added vp9.2 and it works like a charm. As always: Thanks a lot!

ZYinMD commented 4 years ago

When people visit youtube from chrome, and click on a video, and hand pick the highest resolution, (for instance 1080 60fps), is there a good chance youtube would arbitrarily decide to serve webm?

If yes, maybe we could have an flag prefer-website-decision, which works like this: "choose the highest resolution, then accept the file format chosen by the website when serving to a desktop browser with good hardware, fast internet, and all codecs". This way, we will get webm if the server believes it's a good choice, no matter the bitrate.

Let's face it, newer codecs are expected to have lower bitrates. That's the goal when they were developed.