R.I.P. Nitter 🪦😭 (...unless?)

[devgaucho]

https://techcrunch.com/2023/06/30/twitter-now-requires-an-account-to-view-tweets/

the nitter crawler will need to be recreated...

[diehard88]

@devgaucho a la putcha, tchê.

[diehard88]

@rupakhetibinit thoughts and prayers

[diehard88]

@animegrafmays thoughts and prayers

[lleewwiiss]

Has anyone figured out any graphql endpoint for retrieving tweets by id? I have a list of like several thousand tweets I'd like to get asap with as little manual work as possible.

Syndication API still works for that:

import requests

id = "1675187969420828672"
url = f"https://cdn.syndication.twimg.com/tweet-result?id={id}"

r = requests.get(url)

data = r.json()

print(data["text"])

[Write]

Has anyone figured out any graphql endpoint for retrieving tweets by id? I have a list of like several thousand tweets I'd like to get asap with as little manual work as possible.

https://api.twitter.com/graphql/XjlydVWHFIDaAUny86oh2g/TweetDetail?variables=%7B%22focalTweetId%22%3A% + statusId + %22,%22with_rux_injections%22%3Atrue,%22includePromotedContent%22%3Afalse,%22withCommunity%22%3Afalse,%22withQuickPromoteEligibilityTweetFields%22%3Afalse,%22withBirdwatchNotes%22%3Afalse,%22withSuperFollowsUserFields%22%3Afalse,%22withDownvotePerspective%22%3Afalse,%22withReactionsMetadata%22%3Afalse,%22withReactionsPerspective%22%3Afalse,%22withSuperFollowsTweetFields%22%3Afalse,%22withVoice%22%3Atrue,%22withV2Timeline%22%3Atrue%7D&features=%7B%22responsive_web_twitter_blue_verified_badge_is_enabled%22%3Atrue,%22responsive_web_graphql_exclude_directive_enabled%22%3Atrue,%22verified_phone_label_enabled%22%3Afalse,%22responsive_web_graphql_timeline_navigation_enabled%22%3Atrue,%22responsive_web_graphql_skip_user_profile_image_extensions_enabled%22%3Afalse,%22tweetypie_unmention_optimization_enabled%22%3Atrue,%22vibe_api_enabled%22%3Atrue,%22responsive_web_edit_tweet_api_enabled%22%3Atrue,%22graphql_is_translatable_rweb_tweet_is_translatable_enabled%22%3Afalse,%22view_counts_everywhere_api_enabled%22%3Atrue,%22longform_notetweets_consumption_enabled%22%3Atrue,%22tweet_awards_web_tipping_enabled%22%3Afalse,%22freedom_of_speech_not_reach_fetch_enabled%22%3Afalse,%22standardized_nudges_misinfo%22%3Atrue,%22tweet_with_visibility_results_prefer_gql_limited_actions_policy_enabled%22%3Afalse,%22interactive_text_enabled%22%3Atrue,%22responsive_web_text_conversations_enabled%22%3Afalse,%22longform_notetweets_richtext_consumption_enabled%22%3Afalse,%22responsive_web_enhance_cards_enabled%22%3Atrue%7D

[worstperson]

This works for me for UserTweets

#!/bin/sh

[ -z "$1" ] && echo "USAGE: $(basename $0) username_to_search" && exit

USERNAME=$1

echo "Searching..."

ACCESS_TOKEN=$(curl -s -X POST "https://api.twitter.com/oauth2/token?grant_type=client_credentials" -u "CjulERsDeqhhjSme66ECg:IQWdVyqFxghAtURHGeGiWAsmCAGmdW3WmbEx6Hck" | jq .access_token | cut -d '"' -f 2)

GUEST_TOKEN=$(curl -s -X POST -H "Authorization: Bearer $ACCESS_TOKEN" "https://api.twitter.com/1.1/guest/activate.json" | jq .guest_token | cut -d '"' -f 2)

USER_ID=$(curl -s -X GET -H "Authorization: Bearer $ACCESS_TOKEN" \
-H "x-guest-token: $GUEST_TOKEN" \
-H "x-twitter-active-user: yes" \
-H "Referer: https://twitter.com/" \
"https://api.twitter.com/graphql/oUZZZ8Oddwxs8Cd3iW3UEA/UserByScreenName?variables=%7B%22screen_name%22%3A%22$USERNAME%22%2C%22withSafetyModeUserFields%22%3Atrue%7D&features=%7B%22hidden_profile_likes_enabled%22%3Afalse%2C%22responsive_web_graphql_exclude_directive_enabled%22%3Atrue%2C%22verified_phone_label_enabled%22%3Afalse%2C%22subscriptions_verification_info_verified_since_enabled%22%3Atrue%2C%22highlights_tweets_tab_ui_enabled%22%3Atrue%2C%22creator_subscriptions_tweet_preview_api_enabled%22%3Atrue%2C%22responsive_web_graphql_skip_user_profile_image_extensions_enabled%22%3Afalse%2C%22responsive_web_graphql_timeline_navigation_enabled%22%3Atrue%7D" \
| jq -r '.data.user.result.rest_id')

curl -s -o tweets.txt -X GET -H "Authorization: Bearer $ACCESS_TOKEN" \
-H "x-guest-token: $GUEST_TOKEN" \
-H "x-twitter-active-user: yes" \
-H "Referer: https://twitter.com/" \
"https://api.twitter.com/graphql/rCpYpqplOq3UJ2p6Oxy3tw/UserTweets?variables=%7B%22userId%22%3A%22$USER_ID%22,%22count%22%3A10,%22includePromotedContent%22%3Atrue,%22withQuickPromoteEligibilityTweetFields%22%3Atrue,%22withSuperFollowsUserFields%22%3Atrue,%22withDownvotePerspective%22%3Afalse,%22withReactionsMetadata%22%3Afalse,%22withReactionsPerspective%22%3Afalse,%22withSuperFollowsTweetFields%22%3Afalse,%22withVoice%22%3Afalse,%22withV2Timeline%22%3Afalse%7D&features=%7B%22responsive_web_twitter_blue_verified_badge_is_enabled%22%3Atrue,%22responsive_web_graphql_exclude_directive_enabled%22%3Afalse,%22verified_phone_label_enabled%22%3Afalse,%22responsive_web_graphql_timeline_navigation_enabled%22%3Atrue,%22responsive_web_graphql_skip_user_profile_image_extensions_enabled%22%3Afalse,%22tweetypie_unmention_optimization_enabled%22%3Atrue,%22vibe_api_enabled%22%3Atrue,%22responsive_web_edit_tweet_api_enabled%22%3Atrue,%22graphql_is_translatable_rweb_tweet_is_translatable_enabled%22%3Atrue,%22view_counts_everywhere_api_enabled%22%3Atrue,%22longform_notetweets_consumption_enabled%22%3Atrue,%22freedom_of_speech_not_reach_appeal_label_enabled%22%3Afalse,%22standardized_nudges_misinfo%22%3Atrue,%22tweet_with_visibility_results_prefer_gql_limited_actions_policy_enabled%22%3Afalse,%22interactive_text_enabled%22%3Atrue,%22responsive_web_text_conversations_enabled%22%3Afalse,%22responsive_web_enhance_cards_enabled%22%3Afalse%7D"

I dumped json to file tweets.txt rather than parse. Got it from here before they switched to using user's auth tokens: https://github.com/Spicadox/auto-twitter-space/blob/master/index.py

[ScamCast]

Updated: https://github.com/zedeus/nitter/issues/919#issuecomment-1619298877

[Write]

Updated: #919 (comment)

Loving your poor_man_nitter.py

[Write]

Working for me thanks a lot.

This works for me for UserTweets

#!/bin/sh

[ -z "$1" ] && echo "USAGE: $(basename $0) username_to_search" && exit

USERNAME=$1

echo "Searching..."

ACCESS_TOKEN=$(curl -s -X POST "https://api.twitter.com/oauth2/token?grant_type=client_credentials" -u "CjulERsDeqhhjSme66ECg:IQWdVyqFxghAtURHGeGiWAsmCAGmdW3WmbEx6Hck" | jq .access_token | cut -d '"' -f 2)

GUEST_TOKEN=$(curl -s -X POST -H "Authorization: Bearer $ACCESS_TOKEN" "https://api.twitter.com/1.1/guest/activate.json" | jq .guest_token | cut -d '"' -f 2)

USER_ID=$(curl -s -X GET -H "Authorization: Bearer $ACCESS_TOKEN" \
-H "x-guest-token: $GUEST_TOKEN" \
-H "x-twitter-active-user: yes" \
-H "Referer: https://twitter.com/" \
"https://api.twitter.com/graphql/oUZZZ8Oddwxs8Cd3iW3UEA/UserByScreenName?variables=%7B%22screen_name%22%3A%22$USERNAME%22%2C%22withSafetyModeUserFields%22%3Atrue%7D&features=%7B%22hidden_profile_likes_enabled%22%3Afalse%2C%22responsive_web_graphql_exclude_directive_enabled%22%3Atrue%2C%22verified_phone_label_enabled%22%3Afalse%2C%22subscriptions_verification_info_verified_since_enabled%22%3Atrue%2C%22highlights_tweets_tab_ui_enabled%22%3Atrue%2C%22creator_subscriptions_tweet_preview_api_enabled%22%3Atrue%2C%22responsive_web_graphql_skip_user_profile_image_extensions_enabled%22%3Afalse%2C%22responsive_web_graphql_timeline_navigation_enabled%22%3Atrue%7D" \
| jq -r '.data.user.result.rest_id')

curl -s -o tweets.txt -X GET -H "Authorization: Bearer $ACCESS_TOKEN" \
-H "x-guest-token: $GUEST_TOKEN" \
-H "x-twitter-active-user: yes" \
-H "Referer: https://twitter.com/" \
"https://api.twitter.com/graphql/rCpYpqplOq3UJ2p6Oxy3tw/UserTweets?variables=%7B%22userId%22%3A%22$USER_ID%22,%22count%22%3A10,%22includePromotedContent%22%3Atrue,%22withQuickPromoteEligibilityTweetFields%22%3Atrue,%22withSuperFollowsUserFields%22%3Atrue,%22withDownvotePerspective%22%3Afalse,%22withReactionsMetadata%22%3Afalse,%22withReactionsPerspective%22%3Afalse,%22withSuperFollowsTweetFields%22%3Afalse,%22withVoice%22%3Afalse,%22withV2Timeline%22%3Afalse%7D&features=%7B%22responsive_web_twitter_blue_verified_badge_is_enabled%22%3Atrue,%22responsive_web_graphql_exclude_directive_enabled%22%3Afalse,%22verified_phone_label_enabled%22%3Afalse,%22responsive_web_graphql_timeline_navigation_enabled%22%3Atrue,%22responsive_web_graphql_skip_user_profile_image_extensions_enabled%22%3Afalse,%22tweetypie_unmention_optimization_enabled%22%3Atrue,%22vibe_api_enabled%22%3Atrue,%22responsive_web_edit_tweet_api_enabled%22%3Atrue,%22graphql_is_translatable_rweb_tweet_is_translatable_enabled%22%3Atrue,%22view_counts_everywhere_api_enabled%22%3Atrue,%22longform_notetweets_consumption_enabled%22%3Atrue,%22freedom_of_speech_not_reach_appeal_label_enabled%22%3Afalse,%22standardized_nudges_misinfo%22%3Atrue,%22tweet_with_visibility_results_prefer_gql_limited_actions_policy_enabled%22%3Afalse,%22interactive_text_enabled%22%3Atrue,%22responsive_web_text_conversations_enabled%22%3Afalse,%22responsive_web_enhance_cards_enabled%22%3Afalse%7D"

I dumped json to file tweets.txt rather than parse. Got it from here before they switched to using user's auth tokens: https://github.com/Spicadox/auto-twitter-space/blob/master/index.py

Working thanks a lot !

@ScamCast you should add this one :D

[Write]

btw guys, just changing the bearer const. is enough to make nitter work 100%

Here : https://github.com/zedeus/nitter/blob/dcf73354ff173c0407c62f84ea3bb90a130303c1/src/consts.nim#L5C21-L5C21 (Just figured there's all grapheQL endpoints here too)

Proof : https://nitter.socialspill.com/

I really don't think this method (which comes from bird.makeup) will hold long time.

[guys, don't even try to scrape this instance, please, I will shutdown it soon anyway]

EDIT : Well it already stopped working, 🤡 not sure if it's nitter not refreshing guest_tokens or something else

I dont' know enough about nitter guys, it stopped working even though it's not my ip that is limited since poor_man_nitter with the same Bearer tokens still work. So... No idea. All graphql returns 404 not found on nitter

[mthmcalixto]

btw guys, just changing the bearer const. is enough to make nitter work 100%

Here : https://github.com/zedeus/nitter/blob/dcf73354ff173c0407c62f84ea3bb90a130303c1/src/consts.nim#L5C21-L5C21 (Just figured there's all grapheQL endpoints here too)

~Proof : https://nitter.socialspill.com/~

I really don't think this method (which comes from bird.makeup) will hold long time.

[guys, don't even try to scrape this instance, please, I will shutdown it soon anyway]

EDIT : Well it already stopped working, 🤡 not sure if it's nitter not refreshing guest_tokens or something else

I dont' know enough about nitter guys, it stopped working even though it's not my ip that is limited since poor_man_nitter with the same Bearer tokens still work. So... No idea. All graphql returns 404 not found on nitter

It just stopped, it doesn't work.

[Write]

btw guys, just changing the bearer const. is enough to make nitter work 100% Here : https://github.com/zedeus/nitter/blob/dcf73354ff173c0407c62f84ea3bb90a130303c1/src/consts.nim#L5C21-L5C21 (Just figured there's all grapheQL endpoints here too) ~Proof : https://nitter.socialspill.com/~ I really don't think this method (which comes from bird.makeup) will hold long time. [guys, don't even try to scrape this instance, please, I will shutdown it soon anyway] EDIT : Well it already stopped working, 🤡 not sure if it's nitter not refreshing guest_tokens or something else I dont' know enough about nitter guys, it stopped working even though it's not my ip that is limited since poor_man_nitter with the same Bearer tokens still work. So... No idea. All graphql returns 404 not found on nitter

It just stopped, it doesn't work.

Yeah, I know, I edited my message. But it worked for a while. And I really have no idea why it doesn't work anymore since with the same ip / bearer it still work just fine with the poor_man_nitter script

[unseenlarks]

the strange thing is (well, strange to me, a simple end user; i'm sure the rest of you lot will understand) it didn't just stop... it went from displaying all tweets, to just pinned ones, to 502 errors for any additional accounts looked up (the previously looked up still displayed pinned tweets only, even after reloading in a private browser window), then 503 errors for all.

thank you kindly @Write for this latest effort.

[Write]

Pretty sure we need all endpoints compatible with this Bearer / APps for it to work properly. For now

For now we have some working, such as pNl8WjKAvaegIoVH--FuoQ/UserTweetsAndReplies.

We would need all of them that are compatible with this Bearer / Apps to make it work and replace all the one ntiter currently use : https://github.com/zedeus/nitter/blob/dcf73354ff173c0407c62f84ea3bb90a130303c1/src/consts.nim#L5C21-L5C21

For example one of the most important is SearchTimeline, which we haven't found one yet compatible with this specific Bearer.

[ScamCast]

Inspecting all the requests that the Twitter Android app is making inside an emulator. Getting lots of useful stuff.

Twitter uses some funky domain for their app API. https://na.albtls.t.co/graphql/3JNH4e9dq1BifLxAa3UMWg/UserWithProfileTweetsQueryV2

[rohitrsm83]

Greetings,

I am no way a coder/developer but have been using Nitter for a while and sad to see the recent developments. While looking for solutions, I came across this (I remember using it some years ago). Wonder how it works and if it may be of any help at all:

https://tweettunnel.com/maxabrahms

We can see the tweets, albeit in limited forms and clicking on links does take us to Twitter (and no, I didn't sign in).

My 2 pennies.

Best

[alexvong243f]

im having success scraping the embedded timeline as suggested by @fREAST

https://rss-bridge.org/bridge01/?action=display&bridge=TwitterBridge&context=By+username&u=DailyCaller&format=Html (1 hour cache)

[0] code for the temp fix: dvikan/rss-bridge@c4a82ee

Just in case someone don't know about this: RSS-Bridge (official instance and other public instances) also supports generating rss feeds from other sources, e.g. public telegram channels. It's not the 1st time twitter breaks 3rd party program, so I'd assume it's not the last. Some organisations have telegram channels in addition to twitter accounts, e.g. some news sites.

It is sad that things turn out this way. I avoid tiktok, facebook and related services due to their gross privacy violations and not wanting to support their unethical behaviour. Now others are having accessibility issues.

[Melanpan]

Been using my own local Nitter instance to generate a combined RSS feed of users I want to follow, because fuck using the cluster fuck that's Twitter and their algos.

Anyway, ever since Musk-chan put up the login wall and limits, it stopped working. But very rarely, it will still work and let through a couple of tweets. 4 hours ago, one of my rss feeds suddenly contained 14 new tweets that were made today.

It's interesting behavior but briefly looking over the ~500MB logs didn't make me any wiser. Other than seeing a whole lot of 404s for "UserByScreenName" and ratelimit errors for "https://api.twitter.com/2/timeline/profile/"

[polkaulfield]

SearchTimeline

Pretty sure we need all endpoints compatible with this Bearer / APps for it to work properly. For now

For now we have some working, such as pNl8WjKAvaegIoVH--FuoQ/UserTweetsAndReplies.

We would need all of them that are compatible with this Bearer / Apps to make it work and replace all the one ntiter currently use : https://github.com/zedeus/nitter/blob/dcf73354ff173c0407c62f84ea3bb90a130303c1/src/consts.nim#L5C21-L5C21

For example one of the most important is SearchTimeline, which we haven't found one yet compatible with this specific Bearer.

Try this one: G8jKRx5LiyrRDs5FcsUjsw/SearchTimeline

[polkaulfield]

@Write @ScamCast I found a ton of GraphQL endpoints in the android app. Worth looking: https://gist.github.com/polkaulfield/8c84087fafa476e86f768ad25ec44f7b https://gist.github.com/polkaulfield/adfd371a5157c2fb4b334244b29e094a

[ghost]

@Write I found a ton of GraphQL endpoints in the android app. Worth looking: https://gist.github.com/polkaulfield/8c84087fafa476e86f768ad25ec44f7b https://gist.github.com/polkaulfield/adfd371a5157c2fb4b334244b29e094a

Have you tested them? The the Android Bearer does not work and the endpoints are device-dependent. The Mac Bearer works.

Inserting the above constants in the nitter source code makes only the user search work, for me.

[ScamCast]

Was able to find all these from the Android App. Have all their parameters too.

/graphql/IrbvwBFgqiH56L_CJII-7w/ArticleNudgeDomains
/graphql/xVEzTKg_mLTHubK5ayL0HA/AudioSpaceById
/graphql/FIFngkXiu0mzOdbG9qU_MA/BookmarkTimelineV2
/graphql/vm1ta0KYSj1FAwq6T4ffIQ/CarouselImmersiveVideoExploreMixerTimeline
/graphql/xmOcX9TQM1ehLo0EXRWZCA/CommunitiesMembership
/graphql/83h5UyHZ9wEKBVzALX8R_g/ConversationTimelineV2
/graphql/L4cVwybuGrRMhK3ngEJfYw/CreateRetweet
/graphql/b5FipP_f9ATR0AK26CmI1Q/CreateTweet
/graphql/r1IaAd_GIEunlPjVWVlD_w/DeleteRetweet
/graphql/kZyJ4Q1TNsZNByfrGX7Huw/DeleteTweet
/graphql/lI07N6Otwv1PhnEgXILM7A/FavoriteTweet
/graphql/lFi3xnx0auUUnyG4YwpCNw/GetUserClaims
/graphql/b5iDiSI383CiKnpSEzNHhA/HomeTimeline
/graphql/LBazRYwWLieONU-2qh_OHg/ListByIdQuery
/graphql/BbGLL1ZfMibdFNWlk7a0Pw/ListTimeline
/graphql/SLuxVIfuHyvMY5i6X0LEYw/ModeratedTimeline
/graphql/GCyWhVy_JrieSGYS1wOXjw/PinnedListsPut
/graphql/G8jKRx5LiyrRDs5FcsUjsw/SearchTimeline
/graphql/93WcusqmDycQFW9cKo-omg/TrustedFriendsListQueryMembersByRestId
/graphql/Kybeu8_QPBRgVqOo-NpjsQ/TrustedFriendsListQueryRecommendedMembersByRestId
/graphql/LaVEkyIlCyXrD_QXqWkdYA/TrustedFriendsListsQuery
/graphql/XjlydVWHFIDaAUny86oh2g/TweetDetail
/graphql/ZYKSe-w7KEslx3JhSIk5LA/UnfavoriteTweet
/graphql/oUZZZ8Oddwxs8Cd3iW3UEA/UserByScreenName
/graphql/d_AOVAjuTk4qD3QWf-1tAA/UserProfileModulesQuery
/graphql/oPppcargziU1uDQHAUmH-A/UserResultByIdQuery
/graphql/pNl8WjKAvaegIoVH--FuoQ/UserTweetsAndReplies
/graphql/3JNH4e9dq1BifLxAa3UMWg/UserWithProfileTweetsQueryV2
/graphql/1Giswwqz4Ycw_admXVSVEQ/ViewerClaimsQuery
/graphql/971aiUI0d5D5gDFa9HIq6g/ViewerPinnedLists
/graphql/7Y7BnzBIuXcnn6LDO4H-uQ/ViewerUserQuery

Taking a long time to put them in a readable format. 😪

[polkaulfield]

Can you connect using the guest token? I found a ton of hardcoded endpoints too inside the app.

[polkaulfield]

@Write I found a ton of GraphQL endpoints in the android app. Worth looking: https://gist.github.com/polkaulfield/8c84087fafa476e86f768ad25ec44f7b https://gist.github.com/polkaulfield/adfd371a5157c2fb4b334244b29e094a

Have you tested them? The the Android Bearer does not work and the endpoints are device-dependent. The Mac Bearer works.

Inserting the above constants in the nitter source code makes only the user search work, for me.

Didn't test, but that's what i found looking for com.twitter.graphql.GraphQlOperationRegistry

[ScamCast]

@Write @ScamCast I found a ton of GraphQL endpoints in the android app. Worth looking: https://gist.github.com/polkaulfield/8c84087fafa476e86f768ad25ec44f7b https://gist.github.com/polkaulfield/adfd371a5157c2fb4b334244b29e094a

Nice! I just went through and parsed them all out

https://gist.github.com/ScamCast/2e40befbd1b61c4a80cda2745d4df1f4

[nospace-here]

Could OldTwitter be used as reference?

[12joan]

Ok this seems interesting [...] (Comment)

I've hacked together an Express server that fetches recent tweets using this approach and produces a JSON or RSS feed. PRs very much welcome 😄

https://github.com/12joan/twitter-client

[ghost]

please take a look at #925. it rotates bearer tokens through a list of hardcoded ones. it would be good to add the macos one if anybody has it.

i found that with the session cookies of a random user the instance mostly works (#830 is needed for that). if there is a rate limit refreshing a couple of times usually gets you a different bearer token and resets them

[Write]

Hello everyone, nice works of you guys while I was asleep, so cool to see such a community.

please take a look at #925. it rotates bearer tokens through a list of hardcoded ones. it would be good to add the macos one if anybody has it.

i found that with the session cookies of a random user the instance mostly works (#830 is needed for that). if there is a rate limit refreshing a couple of times usually gets you a different bearer token and resets them

Well, the creator of bird.makeup seems to have one available :

https://git.sr.ht/~cloutier/bird.makeup/tree/master/item/src/BirdsiteLive.Twitter/Tools/TwitterAuthenticationInitializer.cs#L36

Just do

curl -X POST "https://api.twitter.com/oauth2/token?grant_type=client_credentials" -u "3rJOl1ODzm9yZy63FACdg:5jPoQ5kQvMJFDYRNE8bQ4rHuds4xJqhvgNJM4awaE8" to get the macOS bearer

AAAAAAAAAAAAAAAAAAAAAIWCCAAAAAAA2C25AxqI%2BYCS7pdfJKRH8Xh19zA%3D8vpDZzPHaEJhd20MKVWp3UR38YoPpuTX7UD2cVYo3YNikubuxd

You could also add the iPad and iPhone one as I see thoses are missing

I personally tried multiples SearchTimeline endpoint you guys posted but it always returns

{"errors":[{"code":126,"message":"Logged out."}]} for me

[Write]

I'm giving up here guys, I don't have the faith to remove SSL Pinning of iOS app and finding all the endpoints for all platforms.

Good luck everyone and thank you ❤️

[net-tech]

Curious as to what @zedeus thinks of all this

[coda101]

Greetings,

I am no way a coder/developer but have been using Nitter for a while and sad to see the recent developments. While looking for solutions, I came across this (I remember using it some years ago). Wonder how it works and if it may be of any help at all:

https://tweettunnel.com/maxabrahms

We can see the tweets, albeit in limited forms and clicking on links does take us to Twitter (and no, I didn't sign in).

My 2 pennies.

Best

Not sure what this is about, do be cautious

[Apachez-]

@coda101

Seems like tweettunnel.com is using ads which bcloudhost.com is part of.

Ghostery will block these two attempts when visiting tweettunnel.com:

httpx://www.bcloudhost.com/9c88e091a401e3c00259b11fe2e91264/invoke.js

httpx://likenesscollecting.com/be501a9bff1a8da595943ddced3825ed/invoke.js

Checking both tweettunnel.com and bcloudhost.com through PaloAlto urlfiltering (https://urlfiltering.paloaltonetworks.com/query/) brings following result:

URL: tweettunnel.com Category: Social Networking Description: User communities and sites where users interact with each other, post messages, pictures, or otherwise communicate with groups of people. Does not include blogs or personal sites. Example Sites: www.facebook.com, www.twitter.com, www.linkedin.com Category: Low Risk Description: Sites that are not medium or high risk are considered low risk. These sites have displayed benign activity for a minimum of 90 days. The low risk category includes both sites that have a history of only benign activity, and sites found to be malicious in the past, but that have displayed benign activity for at least 90 days. Example Sites: www.google.com, www.schwab.com, www.amazon.com

URL: www.bcloudhost.com Category: Grayware Description: Web content that does not pose a direct security threat but that display other intrusive behavior and tempt the end user to grant remote access or perform other unauthorized actions. Grayware includes illegal activities, criminal activities, rogueware, adware, and other unwanted or unsolicited applications, such as embedded crypto miners, clickjacking or hijackers that change the elements of the browser. Typosquatting domains that do not exhibit maliciousness and are not owned by the targeted domain will be categorized as grayware. Example Sites: `

Easiest way to block access to www.bcloudhost.com and bcloudhost.com without involving a browser extension is to modify your /etc/hosts file (similar exists in windows too) and point both domains to either 0.0.0.0 or 127.0.0.1 or 192.0.2.1 or some other non existent IP-address (as in your browser will not get anything bad from it).

Edit: Made all domains non-clickable.

[Apachez-]

When/If nitter resolves the current issue, is it nitter.net who will get the first commits or do there exist some devsite who is public which runs the latest code?

[EthanC]

I've hacked together an Express server that fetches recent tweets using this approach and produces a JSON or RSS feed. PRs very much welcome 😄

https://github.com/12joan/twitter-client

@12joan Any chance you're publicly hosting this? Sounds like we need to test for rate limits 😉

[Rongronggg9]

Special thanks to @Write's information. I've made the Twitter route of RSSHub works again. It always re-authenticates to get a new session after being rate-limited. It should work well unless this approach or the server IP is banned - the latter may occur on the public demo, deploy a personal instance to get rid of it.

• https://github.com/DIYgod/RSSHub/pull/12754

https://rsshub.app/twitter/user/elonmusk https://rsshub.app/twitter/user/elonmusk/excludeReplies=1 https://rsshub.app/twitter/media/bugcat_capoo_tw https://rsshub.app/twitter/keyword/RSSHub

I personally tried multiples SearchTimeline endpoint you guys posted but it always returns {"errors":[{"code":126,"message":"Logged out."}]} for me

@Write What surprises me is that the keyword route implementing search failed to work in my local tests (403 Forbidden), but it turned out to work well on RSSHub's public demo - I didn't change anything of the route except credentials. It seems that https://twitter.com/i/api/2/search/adaptive.json works in specific conditions which are currently unclear, after being authenticated with your approach.

https://github.com/DIYgod/RSSHub/blob/7e45fe12e959b13458f48c7893f0bb92440a1fe0/lib/v2/twitter/web-api/twitter-api.js#L106-L114

[GhbSmwc]

Anyone making a tool I can save to the wayback machine (WBM)? Complete with the proxy's html featuring links to the original resolution of images and gifs (because I saved only those so far), as well as the tweets? I'm still waiting of when the WBM is able to bypass this.

Saving twitter pages directly now results in "Unverified web crawlers are not allowed" rather than a 404, and according to this: https://twitter.com/kevinmarks/status/1676140926077468672 also blocks googlebot.

[devgaucho]

Special thanks to @Write's information. I've made the Twitter route of RSSHub works again. It always re-authenticates to get a new session after being rate-limited. It should work well unless this approach or the server IP is banned - the latter may occur on the public demo, deploy a personal instance to get rid of it.

• fix(route/twitter): Web API authentication DIYgod/RSSHub#12754

https://rsshub.app/twitter/user/elonmusk https://rsshub.app/twitter/user/elonmusk/excludeReplies=1 https://rsshub.app/twitter/media/bugcat_capoo_tw https://rsshub.app/twitter/keyword/RSSHub

I personally tried multiples SearchTimeline endpoint you guys posted but it always returns {"errors":[{"code":126,"message":"Logged out."}]} for me

@Write What surprises me is that the keyword route implementing search failed to work in my local tests (403 Forbidden), but it turned out to work well on RSSHub's public demo - I didn't change anything of the route except credentials. It seems that https://twitter.com/i/api/2/search/adaptive.json works in specific conditions which are currently unclear, after being authenticated with your approach.

https://github.com/DIYgod/RSSHub/blob/7e45fe12e959b13458f48c7893f0bb92440a1fe0/lib/v2/twitter/web-api/twitter-api.js#L106-L114

nice work @Rongronggg9 👏, can read tweets by user id?

[GhbSmwc]

@devgaucho how do I access older tweets so that I can save them to the WBM? It looks like it only listed the most recent X tweets rather than extending to the first existing tweet. I have a bunch of tweet URLs and how do I make rsshub view a specific tweet?

[ghost]

What surprises me is that the keyword route implementing search failed to work in my local tests (403 Forbidden), but it turned out to work well on RSSHub's public demo

I don't think it works on the demo server right now. ~anything other than RSSHub keyword does not work for me.~ I found some keywords like "nitter" that work, but I think SearchTimeline with session cookie works more reliably

I found that oauth credentials sent to /oauth/token always return the same bearer token, so i opted to hardcode them in #925

in that PR, everything but search works unauthenticated (if you refresh often enough and hit the right bearer token) -- the rest can be made to work by adding a cookie header in config. it seems that per-user rate limits get reset if one resets guest-token, even if the cookie is the same

[devgaucho]

@devgaucho how do I access older tweets so that I can save them to the WBM? It looks like it only listed the most recent X tweets rather than extending to the first existing tweet. I have a bunch of tweet URLs and how do I make rsshub view a specific tweet?

embed url https://web.archive.org/web/20230704212606/https://platform.twitter.com/embed/Tweet.html?id=1675260424109928449

[Rongronggg9]

I don't think it works on the demo server right now. ~anything other than RSSHub keyword does not work for me.~ I found some keywords like "nitter" that work

The demo is behind a load balancer, so each request may be processed by a different server. Still, I am quite unclear about the conditions to make it work.

[GhbSmwc]

@devgaucho

@devgaucho how do I access older tweets so that I can save them to the WBM? It looks like it only listed the most recent X tweets rather than extending to the first existing tweet. I have a bunch of tweet URLs and how do I make rsshub view a specific tweet?

embed url https://web.archive.org/web/20230704212606/https://platform.twitter.com/embed/Tweet.html?id=1675260424109928449

Thanks

[Write]

Special thanks to @Write's information. I've made the Twitter route of RSSHub works again. It always re-authenticates to get a new session after being rate-limited. It should work well unless this approach or the server IP is banned - the latter may occur on the public demo, deploy a personal instance to get rid of it.

* [fix(route/twitter): Web API authentication DIYgod/RSSHub#12754](https://github.com/DIYgod/RSSHub/pull/12754)

https://rsshub.app/twitter/user/elonmusk https://rsshub.app/twitter/user/elonmusk/excludeReplies=1 https://rsshub.app/twitter/media/bugcat_capoo_tw https://rsshub.app/twitter/keyword/RSSHub

I personally tried multiples SearchTimeline endpoint you guys posted but it always returns {"errors":[{"code":126,"message":"Logged out."}]} for me

@Write What surprises me is that the keyword route implementing search failed to work in my local tests (403 Forbidden), but it turned out to work well on RSSHub's public demo - I didn't change anything of the route except credentials. It seems that https://twitter.com/i/api/2/search/adaptive.json works in specific conditions which are currently unclear, after being authenticated with your approach.

https://github.com/DIYgod/RSSHub/blob/7e45fe12e959b13458f48c7893f0bb92440a1fe0/lib/v2/twitter/web-api/twitter-api.js#L106-L114

Awesome work !

Can you enlighten me on the differences between the /i/api/2/search/adaptive.json endpoint and the (for example) G8jKRx5LiyrRDs5FcsUjsw/SearchTimeline endpoint ? Can you provide a CURL example of usage with /i/api/2/search/adaptive.json ?

Also your issue seems kinda related to the one I had when I replaced the Bearer in a Nitter instance, worked perfectly for some times and then only 404 without me being able to understand what was wrong since I used the same Bearer / IP. Anyway.

Also it seems that your /excludeReplies=1 endpoint only shows one tweet for elonmusk, certainly because you're filtering entries from /UserTweetsAndReplies or something like that, but @worstperson (https://github.com/zedeus/nitter/issues/919#issuecomment-1619332131) actually made /UserTweets endpoint works and you can get last 20 tweets without any replies :

[Write]

I've updated @ScamCast's awesome poor_man_nitter.py to add UserTweets endpoint.

import requests, json
from pprint import pprint as pp

headers = {
    "Authorization": "Bearer AAAAAAAAAAAAAAAAAAAAAGHtAgAAAAAA%2Bx7ILXNILCqkSGIzy6faIHZ9s3Q%3DQy97w6SIrzE7lQwPJEYQBsArEE2fC25caFwRBvAGi456G09vGR",
    "x-guest-token": "",
    "x-twitter-active-user": "yes",
    "Referer": "https://twitter.com",
    "user-agent": "",
}

def authenticate():
    resp1 = requests.post("https://api.twitter.com/oauth2/token", params={"grant_type": "client_credentials"}, auth=("CjulERsDeqhhjSme66ECg", "IQWdVyqFxghAtURHGeGiWAsmCAGmdW3WmbEx6Hck")).json()
    headers["Authorization"] = f"Bearer {resp1['access_token']}"
    resp2 = requests.post("https://api.twitter.com/1.1/guest/activate.json", headers=headers).json()
    headers["x-guest-token"] = resp2["guest_token"]
    return {"access_token": resp1['access_token'], "guest_token": resp2["guest_token"]}

def UserByScreenName(username):
    return requests.get("https://api.twitter.com/graphql/oUZZZ8Oddwxs8Cd3iW3UEA/UserByScreenName", headers=headers, params={
        "features": json.dumps({
            "hidden_profile_likes_enabled": False,
            "responsive_web_graphql_exclude_directive_enabled": True,
            "verified_phone_label_enabled": False,
            "subscriptions_verification_info_verified_since_enabled": True,
            "highlights_tweets_tab_ui_enabled": True,
            "creator_subscriptions_tweet_preview_api_enabled": True,
            "responsive_web_graphql_skip_user_profile_image_extensions_enabled": False,
            "responsive_web_graphql_timeline_navigation_enabled": True
        }),
        "variables": json.dumps({
            "screen_name": username,
            "withSafetyModeUserFields": True
        })
    }).json()

def UserResultByIdQuery(userId):
    return requests.get("https://api.twitter.com/graphql/oPppcargziU1uDQHAUmH-A/UserResultByIdQuery", headers=headers, params={
        "features": json.dumps({
            "blue_business_profile_image_shape_enabled": False,
            "verified_phone_label_enabled": False,
            "super_follow_user_api_enabled": False,
            "super_follow_badge_privacy_enabled": False,
            "subscriptions_verification_info_enabled": False,
            "creator_subscriptions_subscription_count_enabled": False,
            "super_follow_exclusive_tweet_notifications_enabled": False
        }),
        "variables": json.dumps({
            "user_id": f"{userId}",
            "rest_id": f"{userId}"
        })
    }).json()

def UserTweetsAndReplies(userId):
    return requests.get("https://api.twitter.com/graphql/pNl8WjKAvaegIoVH--FuoQ/UserTweetsAndReplies", headers=headers, params={
        "features": json.dumps({
            "responsive_web_twitter_blue_verified_badge_is_enabled": True,
            "responsive_web_graphql_exclude_directive_enabled": True,
            "verified_phone_label_enabled": False,
            "responsive_web_graphql_timeline_navigation_enabled": True,
            "responsive_web_graphql_skip_user_profile_image_extensions_enabled": False,
            "tweetypie_unmention_optimization_enabled": True,
            "vibe_api_enabled": True,
            "responsive_web_edit_tweet_api_enabled": True,
            "graphql_is_translatable_rweb_tweet_is_translatable_enabled": True,
            "view_counts_everywhere_api_enabled": True,
            "longform_notetweets_consumption_enabled": True,
            "tweet_awards_web_tipping_enabled": False,
            "freedom_of_speech_not_reach_fetch_enabled": False,
            "standardized_nudges_misinfo": True,
            "tweet_with_visibility_results_prefer_gql_limited_actions_policy_enabled": False,
            "interactive_text_enabled": True,
            "responsive_web_text_conversations_enabled": False,
            "longform_notetweets_richtext_consumption_enabled": False,
            "responsive_web_enhance_cards_enabled":False
        }),
        "variables": json.dumps({
            "userId": f"{user_id}",
            "count": 40,
            "includePromotedContent": True,
            "withCommunity": True,
            "withSuperFollowsUserFields": True,
            "withDownvotePerspective": False,
            "withReactionsMetadata": False,
            "withReactionsPerspective": False,
            "withSuperFollowsTweetFields": True,
            "withVoice": True,
            "withV2Timeline":True
        })
    }).json()

def UserTweets(userId):
    return requests.get("https://api.twitter.com/graphql/rCpYpqplOq3UJ2p6Oxy3tw/UserTweets", headers=headers, params={
        "features": json.dumps({
            "responsive_web_twitter_blue_verified_badge_is_enabled": True,
            "responsive_web_graphql_exclude_directive_enabled": False,
            "verified_phone_label_enabled": False,
            "responsive_web_graphql_timeline_navigation_enabled": True,
            "responsive_web_graphql_skip_user_profile_image_extensions_enabled": False,
            "tweetypie_unmention_optimization_enabled": True,
            "vibe_api_enabled": True,
            "responsive_web_edit_tweet_api_enabled": True,
            "graphql_is_translatable_rweb_tweet_is_translatable_enabled": True,
            "view_counts_everywhere_api_enabled": True,
            "longform_notetweets_consumption_enabled": True,
            "freedom_of_speech_not_reach_appeal_label_enabled": False,
            "tweet_awards_web_tipping_enabled": False,
            "freedom_of_speech_not_reach_fetch_enabled": False,
            "standardized_nudges_misinfo": True,
            "tweet_with_visibility_results_prefer_gql_limited_actions_policy_enabled": False,
            "interactive_text_enabled": True,
            "responsive_web_text_conversations_enabled": False,
            "longform_notetweets_richtext_consumption_enabled": False,
            "responsive_web_enhance_cards_enabled": False
        }),
        "variables": json.dumps({
            "userId": f"{user_id}",
            "count": 40,
            "includePromotedContent": True,
            "withQuickPromoteEligibilityTweetFields": True,
            "withSuperFollowsUserFields": True,
            "withDownvotePerspective": False,
            "withReactionsMetadata": False,
            "withReactionsPerspective": False,
            "withSuperFollowsTweetFields": False,
            "withVoice": False,
            "withV2Timeline": False
        })
    }).json()

def TweetDetail(tweetId):
    return requests.get("https://api.twitter.com/graphql/XjlydVWHFIDaAUny86oh2g/TweetDetail", headers=headers, params={
        "features": json.dumps({
            "rweb_lists_timeline_redesign_enabled": True,
            "creator_subscriptions_tweet_preview_api_enabled": True,
            "responsive_web_twitter_article_tweet_consumption_enabled": False,
            "longform_notetweets_rich_text_read_enabled": True,
            "longform_notetweets_inline_media_enabled": True,
            "responsive_web_media_download_video_enabled": True,
            "responsive_web_enhance_cards_enabled": True,
            "responsive_web_twitter_blue_verified_badge_is_enabled": True,
            "responsive_web_graphql_exclude_directive_enabled": True,
            "verified_phone_label_enabled": False,
            "responsive_web_graphql_timeline_navigation_enabled": True,
            "responsive_web_graphql_skip_user_profile_image_extensions_enabled": False,
            "tweetypie_unmention_optimization_enabled": True,
            "vibe_api_enabled": True,
            "responsive_web_edit_tweet_api_enabled": True,
            "graphql_is_translatable_rweb_tweet_is_translatable_enabled": False,
            "view_counts_everywhere_api_enabled": True,
            "longform_notetweets_consumption_enabled": True,
            "tweet_awards_web_tipping_enabled": False,
            "freedom_of_speech_not_reach_fetch_enabled": False,
            "standardized_nudges_misinfo": True,
            "tweet_with_visibility_results_prefer_gql_limited_actions_policy_enabled": False,
            "interactive_text_enabled": True,
            "responsive_web_text_conversations_enabled": False,
            "longform_notetweets_richtext_consumption_enabled": False
        }),
        "variables": json.dumps({
            "focalTweetId": tweetId,
            "with_rux_injections": True,
            "includePromotedContent": False,
            "withCommunity": False,
            "withQuickPromoteEligibilityTweetFields": False,
            "withBirdwatchNotes": False,
            "withSuperFollowsUserFields": False,
            "withDownvotePerspective": False,
            "withReactionsMetadata": False,
            "withReactionsPerspective": False,
            "withSuperFollowsTweetFields": False,
            "withVoice": True,
            "withV2Timeline": True
        }),
        "fieldToggles": json.dumps({
            "withArticleRichContentState": False
        })
    }).json()

test_username = 'elonmusk'
test_tweet_id = 1675390796718014464

authenticate()

user_id = UserByScreenName(test_username)['data']['user']['result']['rest_id']
tweetsAndReplies = UserTweetsAndReplies(user_id)['data']['user']['result']['timeline_v2']['timeline']['instructions'][1]['entries']
tweets = UserTweets(user_id)['data']['user']['result']['timeline']['timeline']['instructions'][1]['entries']
tweet_thread = TweetDetail(test_tweet_id)['data']['threaded_conversation_with_injections_v2']['instructions'][0]['entries']

print(json.dumps(tweets))
#pp(tweet_thread)

what's funny is, if "withV2Timeline": False, then the API does respect the "count": 40, variable ! I set it to false only for the endpoint I added (/UserTweets). I don't really see any downside to it for now. Executing above code like so :

./poor_man_nitter.py | jq '..|.full_text?|select(length>0)'

Returns magnificiently 40 latest tweets just fine (as it's the default set in the python file), NOT cropped or anything.

I tried to change "count:" variable to an arbitrary high number in UserTweets and can get UP to 155 latest tweets.

[ScamCast]

@Write I've updated @ScamCast's awesome poor_man_nitter.py to add UserTweets endpoint.

I was working on a full python module. Should be able to release it tomorrow.

[Write]

I've updated @ScamCast's awesome poor_man_nitter.py to add UserTweets endpoint.

I was working on a full python module. Should be able to release it tomorrow.

Never too late :)

[Write]

Has someone made any endpoint work with an other "auth" than the iPad one ?

        private readonly (string, string)[] _apiKeys = new[]
        {
            ("IQKbtAYlXLripLGPWd0HUA", "GgDYlkSvaPxGxC4X8liwpUoqKwwr3lCADbz8A7ADU"), // iPhone
            ("3nVuSoBZnx6U4vzUxf5w", "Bcs59EFbbsdF6Sl9Ng71smgStWEGwXXKSjYvPVt7qys"), // Android
            ("CjulERsDeqhhjSme66ECg", "IQWdVyqFxghAtURHGeGiWAsmCAGmdW3WmbEx6Hck"), // iPad
            ("3rJOl1ODzm9yZy63FACdg", "5jPoQ5kQvMJFDYRNE8bQ4rHuds4xJqhvgNJM4awaE8"), // Mac
        };

[mosajjal]

during the rate limit time, I never saw the rate limit error on my iPad and Android phone. Tweetdeck and the web client immediately died. Also leads me to believe that the rate limit is not based on username otherwise my tweetdeck being open would've affected my iPad's Twitter which it didn't.

[BANKA2017]

In my latest commit, I used the consumer_token and consumer_token_secret from the Android client, but after I got the bearer token, I further got the open account. it is not clear what is the difference between the open account and the bearer token.

Except for the SearchTimeline endpoint that needs to be re-adapted, other endpoints can be used directly, endpoints used for the community has became login required.