[twitter] hashtag support

HASJ commented 2 years ago

Checklist

[X] I'm reporting a site feature request
[X] I've verified that I'm running yt-dlp version 2021.12.01. (update instructions)
[X] I've checked that all provided URLs are alive and playable in a browser
[X] I've searched the bugtracker for similar issues including closed ones. DO NOT post duplicates
[X] I've read the guidelines for opening an issue
[X] I've read about sharing account credentials and I'm willing to share it if required

Region

Brazil

Example URLs

https://twitter.com/hashtag/BBCF2_MU

Description

In trying to download a twitter hashtag, yt-dlp falls back to the generic URL handler, instead of collecting the twitter posts with media/video and downloading those.

Lesmiscore commented 2 years ago

to devs: I'd suggest to do this via Nitter instances. Because when I did this privately (not yt-dlp related), my accounts got suspended. So I don't recommend doing this directly from yt-dlp.

Ashish0804 commented 2 years ago

it's just a search request for that hastag.

curl 'https://twitter.com/i/api/2/search/adaptive.json?include_profile_interstitial_type=1&include_blocking=1&include_blocked_by=1&include_followed_by=1&include_want_retweets=1&include_mute_edge=1&include_can_dm=1&include_can_media_tag=1&include_ext_has_nft_avatar=1&skip_status=1&cards_platform=Web-12&include_cards=1&include_ext_alt_text=true&include_quote_count=true&include_reply_count=1&tweet_mode=extended&include_entities=true&include_user_entities=true&include_ext_media_color=true&include_ext_media_availability=true&include_ext_sensitive_media_warning=true&send_error_codes=true&simple_quoted_tweet=true&q=%23ThankfulTuesday&count=20&query_source=hashtag_click&cursor=<redacted>&pc=1&spelling_corrections=1&ext=mediaStats%2ChighlightedLabel%2ChasNftAvatar%2CvoiceInfo%2CsuperFollowMetadata' \
  -H 'authority: twitter.com' \
  -H 'authorization: <redacted> \
  -H 'x-twitter-client-language: en' \
  -H 'x-csrf-token:<redacted>' \
  -H 'x-twitter-auth-type: OAuth2Session' \
  -H 'x-twitter-active-user: yes' \
  -H 'user-agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.110 Safari/537.36' \
  -H 'accept: */*' \
  -H 'sec-gpc: 1' \
  -H 'sec-fetch-site: same-origin' \
  -H 'sec-fetch-mode: cors' \
  -H 'sec-fetch-dest: empty' \
  -H 'referer: https://twitter.com/hashtag/ThankfulTuesday?src=hashtag_click' \
  -H 'accept-language: en-US,en;q=0.9' \
  -H 'cookie: <redacted>
  --compressed

I am not sure if it's something dlp should support tho

pukkandan commented 2 years ago

I am not sure if it's something dlp should support tho

why not? We have other search extractors

yt-dlp / yt-dlp