zedeus / nitter

Alternative Twitter front-end
https://nitter.net
GNU Affero General Public License v3.0
9.75k stars 517 forks source link

R.I.P. Nitter 🪦😭 (...unless?) #919

Closed devgaucho closed 11 months ago

devgaucho commented 1 year ago

https://techcrunch.com/2023/06/30/twitter-now-requires-an-account-to-view-tweets/

the nitter crawler will need to be recreated...

warwick-davis commented 1 year ago

why is everyone crying? all you have to do is make an account (and log it into the instance) now

accounts can be deleted and have limited access, which increases the complexity of the code

yes accounts will deleted after they figure out what the account name is... how, exactly?

orbea commented 1 year ago

why is everyone crying? all you have to do is make an account (and log it into the instance) now

Requires a phone number with tor and having an account at all reduces anonymity.

warwick-davis commented 1 year ago

why is everyone crying? all you have to do is make an account (and log it into the instance) now

Requires a phone number with tor and having an account at all reduces anonymity.

phone numbers are cheap

orbea commented 1 year ago

why is everyone crying? all you have to do is make an account (and log it into the instance) now

Requires a phone number with tor and having an account at all reduces anonymity.

phone numbers are cheap

Is it possible to have an anonymous phone number?

warwick-davis commented 1 year ago

why is everyone crying? all you have to do is make an account (and log it into the instance) now

Requires a phone number with tor and having an account at all reduces anonymity.

phone numbers are cheap

Is it possible to have an anonymous phone number?

smspva.com

Thespikedballofdoom commented 1 year ago

Obviously a borderline illegal phone account spamming implementation isn't likely, but thanks for nitter so much zedeus!

devgaucho commented 1 year ago

why is everyone crying? all you have to do is make an account (and log it into the instance) now

accounts can be deleted and have limited access, which increases the complexity of the code

yes accounts will deleted after they figure out what the account name is... how, exactly?

a simple bot detection script can check the xCsrfToken used in the search form of any nitter instance

after that the account is banned by twitter and the bot is taken offline

changing ip is easy and cheap, but creating verified accounts in an automated way is complex and costly

virtual sms number companies usually reuse the same number and each number can be used to create only 10 different twitter accounts

after any suspicious activity the account is blocked and the only way to unblock it is with a new confirmation of the phone number

the access volume of a bot also tends to be higher than a regular user, this draws attention in the same way that logged in users are often blocked after any suspicious activity

warwick-davis commented 1 year ago

why is everyone crying? all you have to do is make an account (and log it into the instance) now

accounts can be deleted and have limited access, which increases the complexity of the code

yes accounts will deleted after they figure out what the account name is... how, exactly?

a simple bot detection script can check the xCsrfToken used in the search form of any nitter instance

after that the account is banned by twitter and the bot is taken offline

changing ip is easy and cheap, but creating verified accounts in an automated way is complex and costly

virtual sms number companies usually reuse the same number and each number can be used to create only 10 different twitter accounts

after any suspicious activity the account is blocked and the only way to unblock it is with a new confirmation of the phone number

the access volume of a bot also tends to be higher than a regular user, this draws attention in the same way that logged in users are often blocked after any suspicious activity

yes, it's an issue for public instances I'll agree. my solution would be disabling search. I've never had any issue with using smspva numbers as they take them off after so many uses (there's a reason why you have to select the application). suspicious activity just means you need multiple accounts to load balance with

markcellus commented 1 year ago

On the smspva.com website:

We do not store numbers longer than 24 hours. If you need to receive SMS for a long time - use the Rental service.

You would need the phone number longer than 24 hours. Basically the lifetime of an account. And thats IF your account isnt flagged for "suspicious activity", which is likely the case for new social media accounts. All new accounts usually have to pass some sort of cleared actions in a certain order over a significant amount of time before your account would be considered "real".

warwick-davis commented 1 year ago

On the smspva.com website:

We do not store numbers longer than 24 hours. If you need to receive SMS for a long time - use the Rental service.

You would need the phone number longer than 24 hours. Basically the lifetime of an account. And thats IF your account isnt flagged for "suspicious activity", which is likely the case for new social media accounts. All new accounts usually have to pass some sort of cleared actions in a certain order over a significant amount of time before your account would be considered "real".

why would you need it for the lifetime of an account? you verify and it's a done deal

fanfare commented 1 year ago

archive.is still works. perhaps they have a pool of logged-in accounts for scraping purposes

warwick-davis commented 1 year ago

archive.is still works. perhaps they have a pool of logged-in accounts for scraping purposes

they do for reddit

orbea commented 1 year ago

smspva.com

I get this when trying to sign up with tor.

Sorry, you cannot register on our website. It was noticed that your e-mail address was used to send spam. If you have never been involved in spam, then your IP-address was previously used by spammers. Ask your internet provider to change your IP-address, and you will be able to register on our website.

Does this also require a payment method which reduce anonymity again....

markcellus commented 1 year ago

why would you need it for the lifetime of an account?

Lol you must be new to scraping. Just because you verify your account with an sms once doesnt mean they wont ask again. :)

devgaucho commented 1 year ago

why would you need it for the lifetime of an account?

Lol you must be new to scraping. Just because you verify your account with an sms once doesnt mean they wont ask again. :)

yes, and with the same number 🙃

warwick-davis commented 1 year ago

why would you need it for the lifetime of an account?

Lol you must be new to scraping. Just because you verify your account with an sms once doesnt mean they wont ask again. :)

actually i'm quite experienced, this has never happened to me once. maybe don't be so cocky? they ask again I switch accounts

warwick-davis commented 1 year ago

smspva.com

I get this when trying to sign up with tor.

Sorry, you cannot register on our website. It was noticed that your e-mail address was used to send spam. If you have never been involved in spam, then your IP-address was previously used by spammers. Ask your internet provider to change your IP-address, and you will be able to register on our website.

Does this also require a payment method which reduce anonymity again....

if you consider monero or bitcoin non-anonymous...

YJSoft commented 1 year ago

In conclusion...

  1. Twitter disabled anonymous access of data
  2. Using accounts also highly likely not works because Twitter will suspend account if its behavior is suspicious(Public instances like nitter.net will become suspended within hour I guess?)
  3. Tricking as bots also not works(as reported)
  4. Scrapping from embedded tweet/timeline might work

About 2(using accounts) Since this change is intented for blocking access of scrappers, If some random account(verified or not) access lots of tweets/account in short time, they might decide that account is scrapping tweets and require re-verify of phone number, captcha, etc. No one knows since It's 100% up to twitter(or elon)

warwick-davis commented 1 year ago

so load balance accounts. all this means is that it costs more to run an instance depending on how much activity there is

devgaucho commented 1 year ago

why would you need it for the lifetime of an account?

Lol you must be new to scraping. Just because you verify your account with an sms once doesnt mean they wont ask again. :)

actually i'm quite experienced, this has never happened to me once. maybe don't be so cocky? they ask again I switch accounts

yes, the most logical way is for nitter to support login data or cookies like youtube-dl and yt-dlp do

in any case nitter will have to be rewritten to support these features

warwick-davis commented 1 year ago

why would you need it for the lifetime of an account?

Lol you must be new to scraping. Just because you verify your account with an sms once doesnt mean they wont ask again. :)

actually i'm quite experienced, this has never happened to me once. maybe don't be so cocky? they ask again I switch accounts

yes, the most logical way is for nitter to support login data or cookies like youtube-dl and yt-dlp do

in any case nitter will have to be rewritten to support these features

agreed. a user could provide their own account but it would compromise privacy

AlyoshaVasilieva commented 1 year ago

I'll note here that the Twitter rate-limit currently seems to be set incredibly low. I'm getting 429 errors browsing my timeline totally normally on my real account.

warwick-davis commented 1 year ago

I'll note here that the Twitter rate-limit currently seems to be set incredibly low. I'm getting 429 errors browsing my timeline totally normally on my real account.

are you on a datacenter vpn?

AlyoshaVasilieva commented 1 year ago

No, residential, accessing using browser without proxy/VPN. According to headers sent by Twitter the rate-limit is currently 50 requests to the TweetDetail endpoint per 15 minutes. Loading a tweet or loading more replies to a tweet both hit it.

OIRNOIR commented 1 year ago

https://platform.twitter.com/embed/Tweet.html?id=1674942336583757825 Elon says it will be unlocked soon, I really hope so.

unseenlarks commented 1 year ago

https://platform.twitter.com/embed/Tweet.html?id=1674942336583757825 Elon says it will be unlocked soon, I really hope so.

thank you for the update. alternatives like bird.makeup, nitter, et al. are so important to have. thank you again to all developers.

rupakhetibinit commented 1 year ago

thoughts?

a02ae1feb7da5958bf7ae451012fad1ab34a974910511745bdcf80f674b8d452

temporary measure, just like the special 3 day military operation in ukraine.

asdf8dfafjk commented 1 year ago

Personally I don't mind twitter making money as long as it supports 2SLGBTQIA++++

Philippe734 commented 1 year ago

Use this amazing nitter service since the beginning. It was so easy to read tweet, it was my daily reads. Without it, I wouldn't have read so much on twitter and I wouldn't have accumulated so much knowledge. Nitter will miss a lot for me.

longyap commented 1 year ago

its so over bros

Nitrousoxide commented 1 year ago

thoughts? a02ae1feb7da5958bf7ae451012fad1ab34a974910511745bdcf80f674b8d452

temporary measure, just like the special 3 day military operation in ukraine.

Doing it hours before the end of the month after Musk has had a history of not paying contractors/suppliers screams that a major host for twitter is shutting them off at the start of July and "oh shit we have to turn stuff off or we'll not be able to keep the site up"

stjdm commented 1 year ago

@Nitrousoxide There might be something in there. Just as of now Twitter web is also down.

D-VR commented 1 year ago

@Nitrousoxide There might be something in there. Just as of now Twitter web is also down.

not just Twitter Web. iOS App also hit.

"rate limit exceeded" since +20 minutes, can't refresh timelines any longer

I think at this point it can be said that Twitter is too unstable to build any kind of software for it.

Even the people who are paying 10k's $ per month for api access get a completely unreliable and buggy service, and 0 support by Twitter.

for you enjoyment: Everything is fine

And developers shelling out for Twitter's Enterprise API Plan, which starts at $42,000 per month, are experiencing sudden outages and disappointing service considering the money they're paying.

"Everything used to work fine before we started paying half a million per year," shared one developer in a private Twitter developer group chat shared with Mashable.

UmBottesWillen commented 1 year ago

note the Android app still works without login, which means the "anonymous flow" is still possible:

https://github.com/4cq2/mech/tree/main/twitter

Or it means that the android app by default tracks as if you were logged in.

michaelkaufman commented 1 year ago

Libreddit gone tomorrow. Wonder how long Invidious and FreeTube will last?

At least reddit still has the old.reddit.com. Which is a pretty descent front-end (as long as you don't care with privacy), with twitter, you are pretty much forced to use that horrible garbage interface. But yeah, this data rush pretty much will make all platforms to garden walled themselves.

What do you mean when you say "as long as you dont care about privacy" with regards to old.reddit?

BANKA2017 commented 1 year ago

note the Android app still works without login, which means the "anonymous flow" is still possible: https://github.com/4cq2/mech/tree/main/twitter

Or it means that the android app by default tracks as if you were logged in.

The principle of Android app anonymous browsing is to register a hidden account with _LO_ prefix screen_name when it is opened for the first time.

And ONLY replacing the bearer token and endpoints will NOT solve the problem.

kloczek commented 1 year ago

Is it any other way to browse accounts which I'm blocked without create another account? (which is IMO stupid) 🤔

Azzurite commented 1 year ago

Is it any other way to browse accounts which I'm blocked without create another account? (which is IMO stupid) 🤔

I wrote myself this script: https://greasyfork.org/en/scripts/469910-twitter-com-to-embed

It rewrites pages to their embed widget URL, which doesn't need login. The relevant URLs to use yourself are

Those could potentially be used to fix Nitter as well?

michaeldvinci commented 1 year ago

Can confirm, the above is working well for both tweets and tweets > user feed

ghost commented 1 year ago

User feeds with the above method don't seem to be working for me, all just show "Nothing to see here yet".

XxTriviumxX commented 1 year ago

For those who use rss feeds. When you go to https://rss.app/rss-feed/create-twitter-rss-feed and enter a twitter page URL you want to follow, you can get its content. You simply need to copy the URL and put it in your bookmarks. If you actually pay for the service, you can get the real rss feed though. I only follow 2 people, so its viable for me.

KonoromiHimaries commented 1 year ago

changing ip is easy and cheap, but creating verified accounts in an automated way is complex and costly

Maybe try to use ipfs to scrape in the user end and store some pieces of content on the nitter webpage, like

https://github.com/meehow/2read
yingziwu commented 1 year ago

image https://twitter.com/elonmusk/status/1675187969420828672

notune commented 1 year ago

@yingziwu use this link instead https://platform.twitter.com/embed/Tweet.html?id=1675187969420828672

sunlightspectre commented 1 year ago

image https://twitter.com/elonmusk/status/1675187969420828672

I don't have much more to contribute here, but people wonder how I've lived so long?

My desire (and track record) of outliving everyone I have a grudge with, serious or petty. Cannot recommend it enough.

Really now... I have to imagine that 6000 tweets could be used up loading more/hitting the back button if you're in an area of particularly high engagement (politics, meteorology/local news, active hobby discussions).
And in addition, paying users barely getting anything more than the quote-un-quote freeloaders that were quote-un-quote 'data' 'pillaging...'

This man really is sui generis.

thechesu commented 1 year ago

That script works great thanks.

With those restrictions from Musk, I can see a mass exodus from Twitter.

fanfare commented 1 year ago

reading the comments/replies to musk's announcement was enough to hit 600 posts, my account is now frozen after about 5 minutes

devgaucho commented 1 year ago

ler os comentários/respostas ao anúncio de musk foi suficiente para atingir 600 postagens, minha conta agora está congelada após cerca de 5 minutos

this is a nightmare for any person with ADHD 😱

garoto commented 1 year ago

I guess advertisers pulling out from the platform was more catastrophic than what was implied in the couple interviews with him I've watched. "Running out of money" to pay necessary infrastructure looks like a believable scenario to me at this point since, well, 2023 mega-corps and ad agencies aren't really on-board with current twitter management and its "controversial" policies.

But on the flip side, instagram seems to be doing just fine sitting behind an authentication wall, so what do I know.

edurur commented 1 year ago

FYI TweetDeck works without any "significant" problems.