search

ciderpunx / twitrssme

TwitRSS.me: Tool to make Twittter timelines and searches into RSS feeds
https://twitrss.me
GNU General Public License v2.0
358 stars 58 forks source link

Twitter update #92

Closed crockpoot closed 4 years ago

crockpoot commented 4 years ago

Twitter updated something and broke the app. Please fix before my backlog gets to nightmarish again.

creideiki commented 4 years ago

Probably has something to do with the old web interface shutting down yesterday, which made the GoodTwitter browser extension stop working: https://twitter.com/ZusorOW/status/1258885451055800320

ciderpunx commented 4 years ago

Thanks for letting me know folks, the mobile_twitter_to_rss.pl version looks to be working for now, I'll redirect traffic n the site there for now - prob not going to be able to look at this properly today.

votemp commented 4 years ago

Possibly relevant fix: https://github.com/twintproject/twint/pull/774/commits/1ad346b38e9574544011e96cada69b7091016eb1

dperelman commented 4 years ago

The fix @votemp linked worked for me. I made a small PR that adds that header which works for me running locally.

crockpoot commented 4 years ago

Thanks for the speedy fix!

pozirk commented 4 years ago

Is it broken again?.. :(

dperelman commented 4 years ago

Yeah, looks like Twitter closed that loophole pretty quickly. twintproject/twint appears to be broken as well. Either that or I can't figure out how to use it.

kalaklanar commented 4 years ago

mobile_twitter_to_rss.pl is still working on my install

ciderpunx commented 4 years ago

Ah, that didn't last long. I've reverted to using mobile_twitter_to_rss.pl on the web service. I'm currently getting "Too many requests" errors, but that could just be while caches warm up. Will see,,,

dperelman commented 4 years ago

As twint is having the same issue, keeping an eye on twintproject/twint#756 and twintproject/twint#782 may help... but so far they have nothing.

crockpoot commented 4 years ago

Dang, I was hoping this wasn't the case when my RSS spat out only a dozen new entries :^\

dperelman commented 4 years ago

... and it suddenly started working again for me. Probably temporary...

crockpoot commented 4 years ago

Weird stuff lol

pozirk commented 4 years ago

Not working for me.

rootless4real commented 4 years ago

I was able to get twint to scrape, but not the latest trss. Has anyone else gotten it working?

zedeus commented 4 years ago

The solution is to rewrite part of twitrss from scratch, or use Nitter's rss feeds instead. Right before the UI change I rewrote Nitter to use the new public JSON API, but it's a massive change https://github.com/zedeus/nitter/pull/169

crockpoot commented 4 years ago

The solution is to rewrite part of twitrss from scratch, or use Nitter's rss feeds instead. Right before the UI change I rewrote Nitter to use the new public JSON API, but it's a massive change zedeus/nitter#169

Hope this works; the backlog.... is getting dire.....

zedeus commented 4 years ago

https://nitter.net serves tens of thousands of requests every day, ~10k RSS feeds daily, it works.

crockpoot commented 4 years ago

Any updates? I use the mobile twitter version but it's finicky and I end up not getting everything. 3 weeks now... @ciderpunx please do something, I'm dying here T_T

jkuj6 commented 4 years ago

Seems nitter is the way to go. However id still like to know if this will be fixed for twitrss, @ciderpunx ?

takemeover commented 4 years ago

I didn't realize this was part of the same issue whoops

ciderpunx commented 4 years ago

Hey @jkuj6 Like zedeus said it probably means a complete rewrite. I'm /really/ short of time at present, so no time soon; unless someone has capacity to take that on.

FWIW the mobile version, though it doesn't work on twitrss.me works to the extent that twitter doesn't rate limit you - so you can run an instance of your own.

bmomjian commented 4 years ago

I don't even have a Twitter account to log in, so I switched to using a free account at https://fetchrss.com/ to get a feed. It says the free account only has a 24-hour update rate.

zedeus commented 4 years ago

You can use Nitter without a Twitter account, it has a 10 minute update rate (you can self-host and change it).

bmomjian commented 4 years ago

Thanks @zedeus, I got it working and it seems fine. I was able to remove all the caching stuff I needed when twitrssme wasn't working, so that is a win.

crockpoot commented 4 years ago

I'd use nitter too but I really need the original URLs for media, unfortunately :(

bmomjian commented 4 years ago

I am already processing the RSS file with Perl, so I just changed nitter.net to twitter.com in the URLs,

crockpoot commented 4 years ago

Any updates?

crockpoot commented 4 years ago

Is there ever getting fixed? It's been way too long.

leonstafford commented 4 years ago

It's been way too long.

@crockpoot - as a maintainer of other open source projects, there's no such thing as "way too long".

The source code is all here, so if it's critical for you / worth something to fix and you can't make the changes (or in this case, maybe rewrite from scratch) yourself, you may be able to hire someone to do that for you. Whether you choose to share that fix back to the project and they accept it is another matter.