jasonmayes / Twitter-Post-Fetcher

Fetch your twitter posts without using the new Twitter 1.1 API. Pure JavaScript! By Jason Mayes
http://www.jasonmayes.com/projects/twitterApi
Other
912 stars 233 forks source link

Plugin no longer working #221

Open joao-guerreiro-asp opened 1 year ago

joao-guerreiro-asp commented 1 year ago

It seems that as of today something has changed and the plugin is no longer working.

Can you please advise?

dk-bold commented 1 year ago

It seems requests to syndication.twitter.com return nothing.

jmcahal commented 1 year ago

I'm having the same issue, getting empty returns from syndication.twitter.com.

makc commented 1 year ago

it seems like the data is now directly embedded in the widget html: image

makc commented 1 year ago

maybe because of this

jasonmayes commented 1 year ago

Looks like a lot of changes going on at Twitter. If someone figures out a way to update feel free to submit a PR and I can look into merging.

mpellegrin commented 1 year ago

It looks like the new Twitter widget is calling an URL like https://syndication.twitter.com/srv/timeline-profile/screen-name/TwitterDev where TwitterDev is the @ of the user.

This URL returns a bunch of HTML with a JSON object embedded inside a tag <script id="__NEXT_DATA__" type="application/json"> .

In theory it should be pretty simple to retrieve this URL and then parse it to get the JSON object, and make some of the plugin functions working again (retrieving raw tweets without all the fancy javascript, iframe, etc).

Unfortunately, it looks like the CORS header is not set by this URL, the Twitter widget uses it to fill an iframe so it cannot be a full frontend solution like the current plugin.

kwagyeman commented 1 year ago

Willing to pay a hundred dollars for this to be fixed. Been using this forever on my website. Would love to see it operational again. :)

kguk commented 9 months ago

I've been having a play with the URL e.g. https://syndication.twitter.com/srv/timeline-profile/screen-name/bbc gives the data back here e.g. @bbc:

As a script tag, full of Json

id="__NEXT_DATA__"

And you're right the CORS header is blocking it from browser-side processing. I tried using a CORS proxy, but couldn't find one that worked or that would return any data.

A simple solution is to have a server-side curl to that URL, then read the contents of the saved file, and extract the JSON. Not sure about the performance overhead of this though. I guess you could save the file in a cache, and once the JSON is extracted delete the file.

I wouldn't host such a service, but I might write this as something that wraps around this plugin, that people can host themselves if there's interest.

jasonmayes commented 9 months ago

@kguk What you describe is basically a proxy server which I guess would work but means paying money by someone to write and maintain that proxy server that then serves the files folk need on the JS side to then use without CORS issues.

jasonmayes commented 9 months ago

Also in order for the https://syndication.twitter.com/srv/timeline-profile/screen-name/bbc to render one must be logged in. So your proxy server must be logged in to work with some other twitter account to view twitter streams.