search

yt-dlp / yt-dlp

A feature-rich command-line audio/video downloader
https://discord.gg/H5MNcFW63r
The Unlicense
90.79k stars 7.05k forks source link

Feature: Number of Quote Tweets in json output #5807

Open foremtehan opened 1 year ago

foremtehan commented 1 year ago

DO NOT REMOVE OR SKIP THE ISSUE TEMPLATE

Checklist

Region

USA

Example URLs

https://twitter.com/MunTheShinobi/status/1600009574919962625

Provide a description that is worded well enough to be understood

Is it possible to add number of quotes from a tweet (also number of views)? It's missing from output json -j

image

yt-dlp https://twitter.com/MunTheShinobi/status/1600009574919962625 -j

"id":"1600009362759733248","title":"Uncle to Ruby and The Twins. - This is a genius ad by Apple. \ud83d\udd25\ud83d\udd25\ud83d\udd25\ud83d\udd25\ud83d\udd25","description":"This is a genius ad by Apple. \ud83d\udd25\ud83d\udd25\ud83d\udd25\ud83d\udd25\ud83d\udd25 https:\/\/t.co\/cNsA0MoOml","uploader":"Uncle to Ruby and The Twins.","timestamp":1670306984,"uploader_id":"MunTheShinobi","uploader_url":"https:\/\/twitter.com\/MunTheShinobi","like_count":57568,"repost_count":16210,"comment_count":501,"age_limit":0,"tags":[],"formats":[{"format_id":"hls-256","format_index":null,"url":"https:\/\/video.twimg.com\/ext_tw_video\/1600009362759733248\/pu\/pl\/480x270\/BsvKygB_QJG0OEgM.m3u8?container=fmp4","manifest_url":"https:\/\/video.twimg.com\/ext_tw_video\/1600009362759733248\/pu\/pl\/jDeo2enqjnKfG4Vo.m3u8?tag=12&container=fmp4","tbr":256,"ext":"mp4","fps":null,"protocol":"m3u8_native","preference":null,"quality":null,"width":480,"height":270,"vcodec":"avc1.4d001e","acodec":"mp4a.40.2","dynamic_range":"SDR","video_ext":"mp4","audio_ext":"none","vbr":256,"abr":0,"format":"hls-256 - 480x270","resolution":"480x270","filesize_approx":4587094,"http_headers":{"User-Agent":"Mozilla\/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit\/537.36 (KHTML, like Gecko) Chrome\/93.0.4577.15 Safari\/537.36","Accept":"text\/html,application\/xhtml+xml,application\/xml;q=0.9,*\/*;q=0.8","Accept-Language":"en-us,en;q=0.5","Sec-Fetch-Mode":"navigate"}},{"url":"https:\/\/video.twimg.com\/ext_tw_video\/1600009362759733248\/pu\/vid\/480x270\/6veOyaeeUQKkPJNd.mp4?tag=12","format_id":"http-256","tbr":256,"width":480,"height":270,"protocol":"https","ext":"mp4","video_ext":"mp4","audio_ext":"none","vbr":256,"abr":0,"format":"http-256 - 480x270","resolution":"480x270","dynamic_range":"SDR","filesize_approx":4587094,"http_headers":{"User-Agent":"Mozilla\/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit\/537.36 (KHTML, like Gecko) Chrome\/93.0.4577.15 Safari\/537.36","Accept":"text\/html,application\/xhtml+xml,application\/xml;q=0.9,*\/*;q=0.8","Accept-Language":"en-us,en;q=0.5","Sec-Fetch-Mode":"navigate"}},{"format_id":"hls-832","format_index":null,"url":"https:\/\/video.twimg.com\/ext_tw_video\/1600009362759733248\/pu\/pl\/640x360\/6Rafz6lJ5ubUY52a.m3u8?container=fmp4","manifest_url":"https:\/\/video.twimg.com\/ext_tw_video\/1600009362759733248\/pu\/pl\/jDeo2enqjnKfG4Vo.m3u8?tag=12&container=fmp4","tbr":832,"ext":"mp4","fps":null,"protocol":"m3u8_native","preference":null,"quality":null,"width":640,"height":360,"vcodec":"avc1.4d001f","acodec":"mp4a.40.2","dynamic_range":"SDR","video_ext":"mp4","audio_ext":"none","vbr":832,"abr":0,"format":"hls-832 - 640x360","resolution":"640x360","filesize_approx":14908055,"http_headers":{"User-Agent":"Mozilla\/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit\/537.36 (KHTML, like Gecko) Chrome\/93.0.4577.15 Safari\/537.36","Accept":"text\/html,application\/xhtml+xml,application\/xml;q=0.9,*\/*;q=0.8","Accept-Language":"en-us,en;q=0.5","Sec-Fetch-Mode":"navigate"}},{"url":"https:\/\/video.twimg.com\/ext_tw_video\/1600009362759733248\/pu\/vid\/640x360\/ZcAmAei-0GTMKxrY.mp4?tag=12","format_id":"http-832","tbr":832,"width":640,"height":360,"protocol":"https","ext":"mp4","video_ext":"mp4","audio_ext":"none","vbr":832,"abr":0,"format":"http-832 - 640x360","resolution":"640x360","dynamic_range":"SDR","filesize_approx":14908055,"http_headers":{"User-Agent":"Mozilla\/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit\/537.36 (KHTML, like Gecko) Chrome\/93.0.4577.15 Safari\/537.36","Accept":"text\/html,application\/xhtml+xml,application\/xml;q=0.9,*\/*;q=0.8","Accept-Language":"en-us,en;q=0.5","Sec-Fetch-Mode":"navigate"}},{"format_id":"hls-2176","format_index":null,"url":"https:\/\/video.twimg.com\/ext_tw_video\/1600009362759733248\/pu\/pl\/1280x720\/GFW9Msh8ckd_HZ3V.m3u8?container=fmp4","manifest_url":"https:\/\/video.twimg.com\/ext_tw_video\/1600009362759733248\/pu\/pl\/jDeo2enqjnKfG4Vo.m3u8?tag=12&container=fmp4","tbr":2176,"ext":"mp4","fps":null,"protocol":"m3u8_native","preference":null,"quality":null,"width":1280,"height":720,"vcodec":"avc1.640020","acodec":"mp4a.40.2","dynamic_range":"SDR","video_ext":"mp4","audio_ext":"none","vbr":2176,"abr":0,"format":"hls-2176 - 1280x720","resolution":"1280x720","filesize_approx":38990299,"http_headers":{"User-Agent":"Mozilla\/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit\/537.36 (KHTML, like Gecko) Chrome\/93.0.4577.15 Safari\/537.36","Accept":"text\/html,application\/xhtml+xml,application\/xml;q=0.9,*\/*;q=0.8","Accept-Language":"en-us,en;q=0.5","Sec-Fetch-Mode":"navigate"}},{"url":"https:\/\/video.twimg.com\/ext_tw_video\/1600009362759733248\/pu\/vid\/1280x720\/odDbaCXr4zhCUw_n.mp4?tag=12","format_id":"http-2176","tbr":2176,"width":1280,"height":720,"protocol":"https","ext":"mp4","video_ext":"mp4","audio_ext":"none","vbr":2176,"abr":0,"format":"http-2176 - 1280x720","resolution":"1280x720","dynamic_range":"SDR","filesize_approx":38990299,"http_headers":{"User-Agent":"Mozilla\/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit\/537.36 (KHTML, like Gecko) Chrome\/93.0.4577.15 Safari\/537.36","Accept":"text\/html,application\/xhtml+xml,application\/xml;q=0.9,*\/*;q=0.8","Accept-Language":"en-us,en;q=0.5","Sec-Fetch-Mode":"navigate"}}],"subtitles":[],"thumbnails":[{"id":"thumb","url":"https:\/\/pbs.twimg.com\/ext_tw_video_thumb\/1600009362759733248\/pu\/img\/XVhFQivj75H_YxxV.jpg?name=thumb","width":150,"height":150,"resolution":"150x150"},{"id":"small","url":"https:\/\/pbs.twimg.com\/ext_tw_video_thumb\/1600009362759733248\/pu\/img\/XVhFQivj75H_YxxV.jpg?name=small","width":680,"height":383,"resolution":"680x383"},{"id":"medium","url":"https:\/\/pbs.twimg.com\/ext_tw_video_thumb\/1600009362759733248\/pu\/img\/XVhFQivj75H_YxxV.jpg?name=medium","width":1200,"height":675,"resolution":"1200x675"},{"id":"large","url":"https:\/\/pbs.twimg.com\/ext_tw_video_thumb\/1600009362759733248\/pu\/img\/XVhFQivj75H_YxxV.jpg?name=large","width":1280,"height":720,"resolution":"1280x720"},{"id":"orig","url":"https:\/\/pbs.twimg.com\/ext_tw_video_thumb\/1600009362759733248\/pu\/img\/XVhFQivj75H_YxxV.jpg?name=orig","width":1280,"height":720,"resolution":"1280x720"}],"duration":139.987,"display_id":"1600009574919962625","_old_archive_ids":["twitter 1600009574919962625"],"webpage_url":"https:\/\/twitter.com\/MunTheShinobi\/status\/1600009574919962625","original_url":"https:\/\/twitter.com\/MunTheShinobi\/status\/1600009574919962625","webpage_url_basename":"1600009574919962625","webpage_url_domain":"twitter.com","extractor":"twitter","extractor_key":"Twitter","playlist":null,"playlist_index":null,"thumbnail":"https:\/\/pbs.twimg.com\/ext_tw_video_thumb\/1600009362759733248\/pu\/img\/XVhFQivj75H_YxxV.jpg?name=orig","fulltitle":"Uncle to Ruby and The Twins. - This is a genius ad by Apple. \ud83d\udd25\ud83d\udd25\ud83d\udd25\ud83d\udd25\ud83d\udd25","duration_string":"2:19","upload_date":"20221206","requested_subtitles":null,"_has_drm":null,"url":"https:\/\/video.twimg.com\/ext_tw_video\/1600009362759733248\/pu\/vid\/1280x720\/odDbaCXr4zhCUw_n.mp4?tag=12","format_id":"http-2176","tbr":2176,"width":1280,"height":720,"protocol":"https","ext":"mp4","video_ext":"mp4","audio_ext":"none","vbr":2176,"abr":0,"format":"http-2176 - 1280x720","resolution":"1280x720","dynamic_range":"SDR","filesize_approx":38990299,"http_headers":{"User-Agent":"Mozilla\/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit\/537.36 (KHTML, like Gecko) Chrome\/93.0.4577.15 Safari\/537.36","Accept":"text\/html,application\/xhtml+xml,application\/xml;q=0.9,*\/*;q=0.8","Accept-Language":"en-us,en;q=0.5","Sec-Fetch-Mode":"navigate"},"epoch":1671042900,"_filename":"Uncle to Ruby and The Twins. - This is a genius ad by Apple. \ud83d\udd25\ud83d\udd25\ud83d\udd25\ud83d\udd25\ud83d\udd25 [1600009362759733248].mp4","filename":"Uncle to Ruby and The Twins. - This is a genius ad by Apple. \ud83d\udd25\ud83d\udd25\ud83d\udd25\ud83d\udd25\ud83d\udd25 [1600009362759733248].mp4","urls":"https:\/\/video.twimg.com\/ext_tw_video\/1600009362759733248\/pu\/vid\/1280x720\/odDbaCXr4zhCUw_n.mp4?tag=12","_type":"video","_version":{"version":"2022.11.11","current_git_head":null,"release_git_head":"8b64402","repository":"yt-dlp\/yt-dlp"}}

Provide verbose output that clearly demonstrates the problem

Complete Verbose Output

/var/www/html $ yt-dlp -vU https://twitter.com/MunTheShinobi/status/1600009574919962625
[debug] Command-line config: ['-vU', 'https://twitter.com/MunTheShinobi/status/1600009574919962625']
[debug] Encodings: locale UTF-8, fs utf-8, pref UTF-8, out utf-8, error utf-8, screen utf-8
[debug] yt-dlp version 2022.11.11 [8b64402] (pip)
[debug] Python 3.10.9 (CPython x86_64 64bit) - Linux-5.15.0-53-generic-x86_64-with (OpenSSL 1.1.1s  1 Nov 2022)
[debug] exe versions: ffmpeg 4.4.1 (fdk,setts), ffprobe 4.4.1
[debug] Optional libraries: sqlite3-2.6.0
[debug] Proxy map: {}
[debug] Loaded 1723 extractors
[debug] Fetching release info: https://api.github.com/repos/yt-dlp/yt-dlp/releases/latest
Latest version: 2022.11.11, Current version: 2022.11.11
yt-dlp is up to date (2022.11.11)
[debug] [twitter] Extracting URL: https://twitter.com/MunTheShinobi/status/1600009574919962625
[twitter] 1600009574919962625: Downloading guest token
[twitter] 1600009574919962625: Downloading JSON metadata
[debug] [twitter] Extracting from video info: 1600009362759733248
[twitter] 1600009574919962625: Downloading m3u8 information
[debug] Sort order given by extractor: res, br, size, proto
[debug] Formats sorted by: hasvid, ie_pref, res, tbr, vbr, abr, filesize, fs_approx, proto, lang, quality, fps, hdr:12(7), vcodec:vp9.2(10), channels, acodec, asr, vext, aext, hasaud, source, id
[debug] Default format spec: bestvideo*+bestaudio/best
[info] 1600009362759733248: Downloading 1 format(s): http-2176
[debug] Invoking http downloader on "https://video.twimg.com/ext_tw_video/1600009362759733248/pu/vid/1280x720/odDbaCXr4zhCUw_n.mp4?tag=12"
[download] Destination: Uncle to Ruby and The Twins. - This is a genius ad by Apple. 🔥🔥🔥🔥🔥 [1600009362759733248].mp4
[download] 100% of   13.04MiB in 00:00:00 at 59.24MiB/s
pukkandan commented 1 year ago

I don't use twitter. Can u explain to me what a "quote tweet" is? Also, do you know any other sites that has similar feature?

bashonly commented 1 year ago

"quote tweet"

It's a comment/repost hybrid. If User A "quote tweets" User B's original tweet, then A's comment doesn't show up in the replies of B's original tweet; instead, B's original tweet is reposted to A's timeline with A's comment on top of it.

Some other sites maybe have some similar features (e.g. facebook's "share to feed" w/description, tiktok's "duets," etc), but I can't think of another site that displays a count specifically for that type of repost. Mastodon doesn't have any official mechanism for this. I haven't tried any of the twitter knock-offs that have been popping up in recent years.

foremtehan commented 1 year ago

Twitter also added total view of a tweet:

image

bashonly commented 1 year ago

Twitter also added total view of a tweet:

IMO we should wait and see if this is just another new "improvement" that's going to be rolled back after ~2 weeks before taking any action to add metadata