ytdl-org / youtube-dl

Command-line program to download videos from YouTube.com and other video sites
http://ytdl-org.github.io/youtube-dl/
The Unlicense
131.22k stars 9.93k forks source link

Average_rating not behaving correctly in match_filter #18734

Open tukruic opened 5 years ago

tukruic commented 5 years ago

Please follow the guide below


Make sure you are using the latest version: run youtube-dl --version and ensure your version is 2019.01.02. If it's not, read this FAQ entry and update. Issues with outdated version will be rejected.

Before submitting an issue make sure you have:

What is the purpose of your issue?

If the purpose of this issue is a bug report, site support request or you are not completely sure provide the full verbose output as follows:

Add the -v flag to your command line you run youtube-dl with (youtube-dl -v <your command line>), copy the whole output and insert it here. It should look similar to one below (replace it with your log inserted between triple ```):

krzysztofs-macbook-pro:~ kk$ youtube-dl -v --match-filter "average_rating > 4.8" "ytsearch10:ai"
[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: ['-v', '--match-filter', 'average_rating > 4.8', 'ytsearch10:ai']
[debug] Encodings: locale UTF-8, fs utf-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2019.01.02
[debug] Python version 3.7.1 (CPython) - Darwin-17.7.0-x86_64-i386-64bit
[debug] exe versions: none
[debug] Proxy map: {}
[youtube:search] query "ai": Downloading page 1
[download] Downloading playlist: ai
[youtube:search] playlist ai: Collected 10 video ids (downloading 10 of them)
[download] Downloading video 1 of 10
[youtube] BlPFE7R8psk: Downloading webpage
[youtube] BlPFE7R8psk: Downloading video info webpage
[debug] Default format spec: best/bestvideo+bestaudio
[debug] Invoking downloader on 'https://r7---sn-u2oxu-f5fk.googlevideo.com/videoplayback?id=o-AAKqTHFlLlMVpDJKZKHxmX4fsL8igFrhreJZiGN9pwCQ&mn=sn-u2oxu-f5fk%2Csn-u2oxu-f5fk&c=WEB&ipbits=0&mm=31%2C29&initcwndbps=830000&shardbypass=yes&ip=79.184.191.139&ms=au%2Crdu&mv=m&dur=754.439&mt=1546564780&expire=1546586551&hcs=yes&requiressl=yes&lmt=1546301468088071&key=yt6&txp=5532432&pl=24&mime=video%2Fmp4&source=youtube&signature=C6FC95D35A72076D82EF898AF7A1FB99919EF002.1DBF031BA9FF5C26430E160B7D2DCFBC8195641A&fvip=8&ratebypass=yes&sparams=dur%2Cei%2Chcs%2Cid%2Cinitcwndbps%2Cip%2Cipbits%2Citag%2Clmt%2Cmime%2Cmm%2Cmn%2Cms%2Cmv%2Cpl%2Cratebypass%2Crequiressl%2Cshardbypass%2Csource%2Cexpire&itag=22&ei=V7UuXKu1Is_wgQfY-rz4Dg'
[download] Destination: AI in 2019-BlPFE7R8psk.mp4
[download] 100% of 193.41MiB in 01:17
[download] Downloading video 2 of 10
[youtube] ajGgd9Ld-Wc: Downloading webpage
[youtube] ajGgd9Ld-Wc: Downloading video info webpage
[debug] Default format spec: best/bestvideo+bestaudio
[debug] Invoking downloader on 'https://r2---sn-u2oxu-fhpe.googlevideo.com/videoplayback?source=youtube&ratebypass=yes&mv=m&dur=889.997&mt=1546564881&lmt=1537507347119271&itag=22&ms=au%2Conr&sparams=dur%2Cei%2Cid%2Cinitcwndbps%2Cip%2Cipbits%2Citag%2Clmt%2Cmime%2Cmm%2Cmn%2Cms%2Cmv%2Cpl%2Cratebypass%2Crequiressl%2Csource%2Cexpire&key=yt6&ipbits=0&mm=31%2C26&mn=sn-u2oxu-fhpe%2Csn-5hnekn7k&ei=prUuXJz7HIXIgAf4zrfYBg&expire=1546586630&c=WEB&id=o-AMkbfujCf-4sBA-LsWMt3Nr5qeUKJQ6STGuR3wVo3t57&fvip=1&initcwndbps=930000&ip=79.184.191.139&requiressl=yes&mime=video%2Fmp4&signature=2A5AAA482669C061C91443F1B4A99C73BDF244A7.888F5B426F4FE747FC047953A4C10E01BD809A87&pl=24'
[download] Destination: How AI can save our humanity _ Kai-Fu Lee-ajGgd9Ld-Wc.mp4
[download] 100% of 46.73MiB in 00:01
[download] Downloading video 3 of 10
[youtube] Pls_q2aQzHg: Downloading webpage
[youtube] Pls_q2aQzHg: Downloading video info webpage
[debug] Default format spec: best/bestvideo+bestaudio
[debug] Invoking downloader on 'https://r6---sn-u2oxu-f5fk.googlevideo.com/videoplayback?mime=video%2Fmp4&ip=79.184.191.139&key=yt6&itag=22&ipbits=0&txp=5531432&ei=rLUuXMnbLYi9gAellJCoBA&lmt=1538778780005228&signature=9043115AECBD7929121BD3E449C9727BED1C6F6E.212FE4190E4424F94FE692402525785C9BDE1BE4&id=o-AORoROtsYT5X0gH0DdVghc5F4cCo267mrGjH7piHb7Dt&sparams=dur%2Cei%2Cid%2Cinitcwndbps%2Cip%2Cipbits%2Citag%2Clmt%2Cmime%2Cmm%2Cmn%2Cms%2Cmv%2Cpl%2Cratebypass%2Crequiressl%2Csource%2Cexpire&initcwndbps=891250&mm=31%2C26&source=youtube&mn=sn-u2oxu-f5fk%2Csn-5hnedn7e&ratebypass=yes&c=WEB&expire=1546586636&fvip=6&pl=24&dur=1222.019&requiressl=yes&mt=1546564881&mv=m&ms=au%2Conr'
[download] Destination: Artificial Intelligence - Mankind's Last Invention-Pls_q2aQzHg.mp4
[download] 100% of 115.46MiB in 00:03
[download] Downloading video 4 of 10
[youtube] _0gUXxZ-ExE: Downloading webpage
[youtube] _0gUXxZ-ExE: Downloading video info webpage
[debug] Default format spec: best/bestvideo+bestaudio
[debug] Invoking downloader on 'https://r2---sn-u2oxu-fhpe.googlevideo.com/videoplayback?initcwndbps=930000&ipbits=0&mt=1546564881&requiressl=yes&mm=31%2C26&ei=srUuXMKILtrB-gbvxJHQCA&fvip=2&signature=03F9E2443E325995349B5EFFB6F957E424427047.C2D2C2A8332ADFEBE0D31EB367AFB8DFA2BBFD24&c=WEB&lmt=1545068744140441&mime=video%2Fmp4&ratebypass=yes&id=o-ADhx1sB6j5CfmUTSkmmE7YK8pPbcRkZtKpOQ-jvYPE6q&expire=1546586642&sparams=dur%2Cei%2Cid%2Cinitcwndbps%2Cip%2Cipbits%2Citag%2Clmt%2Cmime%2Cmm%2Cmn%2Cms%2Cmv%2Cpl%2Cratebypass%2Crequiressl%2Csource%2Cexpire&mn=sn-u2oxu-fhpe%2Csn-5hnednlk&txp=5535432&ms=au%2Conr&source=youtube&mv=m&dur=960.818&itag=22&pl=24&ip=79.184.191.139&key=yt6'
[download] Destination: Global race for AI superiority-_0gUXxZ-ExE.mp4
[download] 100% of 93.18MiB in 00:02
[download] Downloading video 5 of 10
[youtube] YNLC0wJSHxI: Downloading webpage
[youtube] YNLC0wJSHxI: Downloading video info webpage
[debug] Default format spec: best/bestvideo+bestaudio
[debug] Invoking downloader on 'https://r6---sn-u2oxu-f5fd.googlevideo.com/videoplayback?mv=m&mt=1546564973&ms=au%2Conr&source=youtube&pl=24&mn=sn-u2oxu-f5fd%2Csn-5hnekn7s&mm=31%2C26&dur=352.060&requiressl=yes&id=o-AIje6kkM29PtH3r5jm5EGnSScnRZLKqWlALeM7MdesA7&expire=1546586648&ipbits=0&sparams=dur%2Cei%2Cid%2Cinitcwndbps%2Cip%2Cipbits%2Citag%2Clmt%2Cmime%2Cmm%2Cmn%2Cms%2Cmv%2Cpcm2cms%2Cpl%2Cratebypass%2Crequiressl%2Csource%2Cexpire&initcwndbps=963750&signature=BEA060042BE8301A48CA42B1B58CE3C731950A41.E099C792434487B79DFB8219EF783C166511FD32&lmt=1516092900639810&ip=79.184.191.139&pcm2cms=yes&key=yt6&itag=22&mime=video%2Fmp4&fvip=5&ratebypass=yes&c=WEB&ei=t7UuXK71PIip1gLMn5-ABg'
[download] Destination: AI Codes its Own ‘AI Child’  - Artificial Intelligence breakthrough!-YNLC0wJSHxI.mp4
[download] 100% of 39.41MiB in 00:05
[download] Downloading video 6 of 10
[youtube] uNO7QwGZKic: Downloading webpage
[youtube] uNO7QwGZKic: Downloading video info webpage
[debug] Default format spec: best/bestvideo+bestaudio
[debug] Invoking downloader on 'https://r2---sn-u2oxu-f5fe6.googlevideo.com/videoplayback?requiressl=yes&ipbits=0&signature=9BC316787D02BD542513DD3F232113E9AE68AD82.071360A3CCA2949AB830318AE8D73A5436EB360E&pl=24&mime=video%2Fmp4&fvip=2&lmt=1540580178397287&ip=79.184.191.139&expire=1546586659&c=WEB&ratebypass=yes&id=o-AERbPXRYsbFSSVhQRWst2CaPNccwf1TlUDxlnVzEGXbh&mt=1546564973&mn=sn-u2oxu-f5fe6%2Csn-5hne6nsy&mm=31%2C26&txp=5531432&sparams=dur%2Cei%2Cid%2Cinitcwndbps%2Cip%2Cipbits%2Citag%2Clmt%2Cmime%2Cmm%2Cmn%2Cms%2Cmv%2Cpl%2Cratebypass%2Crequiressl%2Csource%2Cexpire&key=yt6&mv=m&dur=1966.730&source=youtube&initcwndbps=778750&itag=22&ms=au%2Conr&ei=w7UuXOrrH8_JgAe5z4qoCw'
[download] Destination: Joe Rogan Goes Balls Deep On A.I.-uNO7QwGZKic.mp4
[download] 100% of 110.21MiB in 00:03
[download] Downloading video 7 of 10
[youtube] 2eVR0i5YAv0: Downloading webpage
[youtube] 2eVR0i5YAv0: Downloading video info webpage
[debug] Default format spec: best/bestvideo+bestaudio
[debug] Invoking downloader on 'https://r1---sn-u2oxu-f5fe6.googlevideo.com/videoplayback?signature=A703C97C2F2984C5AE267D24B571B61141F2B0F2.E55690B4487A7DADB2915F8D2C7186ED267338&lmt=1540167245156324&ratebypass=yes&fvip=4&ei=zLUuXPSZO42wgQebwZj4DA&requiressl=yes&ipbits=0&mt=1546564973&sparams=dur%2Cei%2Cid%2Cinitcwndbps%2Cip%2Cipbits%2Citag%2Clmt%2Cmime%2Cmm%2Cmn%2Cms%2Cmv%2Cpl%2Cratebypass%2Crequiressl%2Csource%2Cexpire&initcwndbps=778750&ip=79.184.191.139&key=yt6&dur=641.474&itag=22&pl=24&source=youtube&mv=m&ms=au%2Conr&mm=31%2C26&mn=sn-u2oxu-f5fe6%2Csn-5hne6n7s&id=o-ABH_cxqpxjqsBvjWFhprN7KcNinrz-g_LJSUnUhUR22I&c=WEB&expire=1546586669&txp=5531432&mime=video%2Fmp4'
[download] Destination: 6 Scariest Things Said by A.I. Robots-2eVR0i5YAv0.mp4
[download] 100% of 41.91MiB in 00:01
[download] Downloading video 8 of 10
[youtube] WSW-5m8lRMs: Downloading webpage
[youtube] WSW-5m8lRMs: Downloading video info webpage
[debug] Default format spec: best/bestvideo+bestaudio
[debug] Invoking downloader on 'https://r7---sn-u2oxu-f5fk.googlevideo.com/videoplayback?ratebypass=yes&c=WEB&lmt=1545435539063909&expire=1546586676&mime=video%2Fmp4&source=youtube&itag=22&ei=1LUuXMveBYns1wKupIPgAg&requiressl=yes&fvip=4&dur=465.351&mt=1546564973&mm=31%2C26&signature=A670E4733F0C5DC68D1750113156E394C2EB165A.E056149B1CF737ADAE6690AB5958B25B698C1670&key=yt6&id=o-AK-5tABpsJ5dMLktksFqMYqj_B--P6g46Vi7zqV8UN0_&sparams=dur%2Cei%2Cid%2Cinitcwndbps%2Cip%2Cipbits%2Citag%2Clmt%2Cmime%2Cmm%2Cmn%2Cms%2Cmv%2Cpl%2Cratebypass%2Crequiressl%2Csource%2Cexpire&ipbits=0&initcwndbps=860000&mv=m&pl=24&txp=4535432&ms=au%2Conr&ip=79.184.191.139&mn=sn-u2oxu-f5fk%2Csn-5hnekn7l'
[download] Destination: A.I. Learns to play Flappy Bird-WSW-5m8lRMs.mp4
[download] 100% of 15.12MiB in 00:00
[download] Downloading video 9 of 10
[youtube] XvzNuw5VjBU: Downloading webpage
[youtube] XvzNuw5VjBU: Downloading video info webpage
[youtube] {22} signature length 40.41, html5 player vflSzU_20
[youtube] {43} signature length 40.41, html5 player vflSzU_20
[youtube] {18} signature length 40.41, html5 player vflSzU_20
[youtube] {36} signature length 40.41, html5 player vflSzU_20
[youtube] {17} signature length 40.41, html5 player vflSzU_20
[youtube] {313} signature length 40.41, html5 player vflSzU_20
[youtube] {271} signature length 40.41, html5 player vflSzU_20
[youtube] {137} signature length 40.41, html5 player vflSzU_20
[youtube] {248} signature length 11.66.1, html5 player vflSzU_20
[youtube] XvzNuw5VjBU: Downloading player https://www.youtube.com/yts/jsbin/player_ias-vflSzU_20/en_US/base.js
[youtube] {136} signature length 40.41, html5 player vflSzU_20
[youtube] {247} signature length 40.41, html5 player vflSzU_20
[youtube] {135} signature length 38.41, html5 player vflSzU_20
[youtube] {244} signature length 40.41, html5 player vflSzU_20
[youtube] {134} signature length 40.41, html5 player vflSzU_20
[youtube] {243} signature length 40.41, html5 player vflSzU_20
[youtube] {133} signature length 40.41, html5 player vflSzU_20
[youtube] {242} signature length 40.41, html5 player vflSzU_20
[youtube] {160} signature length 40.41, html5 player vflSzU_20
[youtube] {278} signature length 40.41, html5 player vflSzU_20
[youtube] {140} signature length 40.41, html5 player vflSzU_20
[youtube] {171} signature length 40.41, html5 player vflSzU_20
[youtube] {249} signature length 40.41, html5 player vflSzU_20
[youtube] {250} signature length 40.41, html5 player vflSzU_20
[youtube] {251} signature length 40.41, html5 player vflSzU_20
[debug] Default format spec: best/bestvideo+bestaudio
[debug] Invoking downloader on 'https://r7---sn-u2oxu-fhpe.googlevideo.com/videoplayback?sparams=dur%2Cei%2Cid%2Cinitcwndbps%2Cip%2Cipbits%2Citag%2Clmt%2Cmime%2Cmm%2Cmn%2Cms%2Cmv%2Cpcm2cms%2Cpl%2Cratebypass%2Crequiressl%2Csource%2Cexpire&fvip=1&mime=video%2Fmp4&txp=5432432&pcm2cms=yes&source=youtube&pl=24&ratebypass=yes&initcwndbps=773750&c=WEB&mm=31%2C26&mn=sn-u2oxu-fhpe%2Csn-5hne6nsk&ipbits=0&requiressl=yes&expire=1546586680&mt=1546564973&mv=m&ei=2LUuXOnwGJGOgQe1g4rgDw&ms=au%2Conr&lmt=1545664860984440&dur=260.713&key=yt6&ip=79.184.191.139&itag=22&id=o-AE6X1GxV6x2EGw9grq-6c5vwF3UfzmSxd3Rwgl2XmAY2&signature=AE3537BE1468A22636E8FE0D5A2DC688ACAC9183.573A56DDCA0A3EADAA1AAB70EC2659160D481345'
[download] Destination: How artificial intelligence will change your world in 2019, for better or worse-XvzNuw5VjBU.mp4
[download] 100% of 37.18MiB in 00:12
[download] Downloading video 10 of 10
[youtube] Dk7h22mRYHQ: Downloading webpage
[youtube] Dk7h22mRYHQ: Downloading video info webpage
[debug] Default format spec: best/bestvideo+bestaudio
[debug] Invoking downloader on 'https://r6---sn-u2oxu-f5fk.googlevideo.com/videoplayback?dur=3085.815&mv=m&source=youtube&ms=au%2Conr&itag=22&initcwndbps=860000&sparams=dur%2Cei%2Cid%2Cinitcwndbps%2Cip%2Cipbits%2Citag%2Clmt%2Cmime%2Cmm%2Cmn%2Cms%2Cmv%2Cpl%2Cratebypass%2Crequiressl%2Csource%2Cexpire&id=o-AETt-6gVSToAIxdxcSBdPywAJntuTwvv0oEaQh7C9OEy&mt=1546564973&mn=sn-u2oxu-f5fk%2Csn-5hne6nsd&mm=31%2C26&ipbits=0&pl=24&signature=6935470D8D8F0D3086FA03AE30F53D635E984818.98AAA38EF9CDB63D8F9F932D041AFEE3B2DA1105&ratebypass=yes&ip=79.184.191.139&ei=7LUuXN7LFPH6gAfchZnQCA&lmt=1529121092479258&c=WEB&expire=1546586700&key=yt6&fvip=4&requiressl=yes&mime=video%2Fmp4'
[download] Destination: The Rise of AI-Dk7h22mRYHQ.mp4
[download] 100% of 365.54MiB in 00:16
[download] Finished downloading playlist: ai

Description of your issue, suggested solution and other information

Basically the average_rating parameter appears to not be working correctly, according to: krzysztofs-macbook-pro:~ kk$ youtube-dl -o '%(title)s.%(average_rating)s.%(ext)s' "ytsearch10:ai" 5 of these videos should not be downloaded( for example last one has average_rating of 4.7812795639)

Lomacar commented 4 years ago

I have the same problem. In the test I am doing only one video is filtered out even though many other videos are below the rating threshold. Youtube-dl knows what the rating is but still doesn't filter correctly. I think it is only accepting integer values (flooring the rating threshold given by the user). Because the one video that it does filter out for me has a rating of 3.6. Anything at or above 4 and the filter catches it, but at 3.9 it passes through.

Lomacar commented 4 years ago

For now a workaround could be to filter for like_count > X & dislike_count < Y. But this should be easily fixed.