Twitter API v1.0 / v1.1 support

[jeffehobbs]

Needs API key authentication...or maybe not. In light of the new API rules from Twitter, will tweet.js require an API key? The way I'm reading the new rules, it seems like Twitter is going to want to know who we all are, which connotes "API key".

~Jeff

[purcell]

To which new rules are you referring? I haven't read of anything.

[purcell]

Found what you're referring to. Will read and respond.

[jeffehobbs]

Right. https://dev.twitter.com/blog/changes-coming-to-twitter-api . "required authentication on every API endpoint".

[jeffehobbs]

Also, I could see "Display Requirements" being a PITA.

[purcell]

I've tweeted 'em to find out. We're likely to need to make changes, but I'd hope that whatever plumbing they use to support their own widgets could also be used by ours... sigh.

[purcell]

No response. Given the general discussion around this topic, I wouldn't say things look good for the future of jquery.tweet.js.

[raideus]

This is probably the best place to get answers: https://dev.twitter.com/discussions

Picked up the following tidbit from a Twitter rep:

"There are no client-side only solutions besides what we offer in our Twitter for Websites family of products at this time. If you want to use the API itself, you'll need to use server-side OAuth. If you want to engage with the platform client-side, you'll need to use the Tweet Button, Follow Button, Embeddable Timelines, Embeddable Tweets, the tweet, follow, favorite, and retweet web intents, and their accompanying javascript events. We're still working on this client-side family of products and there may be further enhancements to the API's auth model in the future that would make client-side development more palatable."

Not sure if that is relevant in any way to jQuery Tweet?

[purcell]

@bootsz Yes, it suggest that jQuery Tweet will no longer be able to work in its current form. However, if Twitter is providing its own widgets which presumably have unauthenticated access to backend data, it's possible that Tweet can make itself look like the official widgets and continue to access that same data.

[raideus]

Got it. In terms of Tweet "making itself look like the official widgets", any idea how that would be accomplished? I've played around with their new widget a bit and it doesn't seem like they allow hardly any customization from the front-end.

[purcell]

@bootsz I just meant from a network point of view -- while the appearance of Tweet wouldn't be identical, it could perhaps be made to make network requests in a manner indistinguishable from the official widget. However, it might still be in violation of the UX guidelines.

[danielkorte]

So just for clarity's sake, right now at it's current state, jquery.tweet.js would not be a solution if say a large chunk of your site's visitors are visiting from the same IP address where rate limiting would occur very often (and very quickly)?

[purcell]

@danielkorte Correct. But neither would Twitter's official widgets be a solution, for the same reason. At least currently. It's unclear what the situation will be with rate limiting after the unauthenticated 1.0 API is disabled.

[StanScates]

Howdy, I'm rather fond of jquery.tweet.js, so due to the impending deprecation of Twitter API v1.0, I created a simple OAuth / caching layer designed to interface with a slightly modified version of jquery.tweet.js. Set up a Twitter App for your feed, use the customized jquery.tweet.js provided and plug your OAuth credentials into the index.php file included.

Hopefully someone will find this useful. I'm converting all of our production sites which currently use jquery.tweet.js to use this henceforth.

https://github.com/StanScates/Tweet.js-Mod

Feel free to contact me with questions/comments. Thanks -Stan

[raideus]

Stan, THANK YOU for this! Looking forward to trying it out.

[StanScates]

Let me know how it goes :smiley_cat:

[purcell]

Hi Stan, that's neat! I'll have a think about how/whether to merge some of that back into this main repo. (Since you didn't start by forking this repo it's not easy to see what changes you've made to tweet.js, but I'm guessing it's limited to the build_api_url function.)

You'd have to be careful that other people didn't point scripts etc at your endpoint -- it'd be very easy to exhaust your twitter credentials' request quota.

-Steve

[StanScates]

Hey Steve, I've just rolled out v1.3, I've made several changes to jquery.tweet.js (feel free to take a look, I would highly value any input you may have), I would certainly be open to forking your repository but I didn't want to jump the gun since this introduces a whole new layer of complexity to your excellent plugin and I didn't want to pollute its simplicity (yet). Please send me an email if you wish and we can discuss how/whether you'd like to package them.

To your second point, although it is theoretically possible, I feel that the effort involved in order to circumvent Same Origin Policy restrictions makes that an unlikely scenario. With that said, I do highly recommend the caching feature, it's enabled by default and very simple.

I very much appreciate your input.

[purcell]

Hi @Mandooox, please file related issues on tweet.js-mod rather than here.

[mohamdio]

hi @purcell , really sorry for that , okay i will delete this comment

but about your plugin , are you will update it or what can we do for [ API 1.1 ] :( ?

and other ask : when search about new API , i found that should any user will use plugin should Create new APP to can use the plugin after update by Stan ... are that right ...?

thanks again for your great work in this plugin.

[purcell]

@Mandooox As discussed earlier in this issue, it's not yet clear whether it will be possible to keep tweet.js working after the old API is disabled. For now, it's still (mostly) working fine.

After the old API is disabled, Twitter will still continue to supply their own (unauthenticated) official tweet widgets, so in theory it would still be possible for tweet.js to use whatever API those official widgets use.

-Steve

[mohamdio]

okay , really i'm not happy with this new API :( , your plugin was perfect and easy to use , and i hope it will be still working..... thanks

[petenorris]

Not being a developer, I'm still unsure if seaofclouds twitter feed is going to work or not. Is there no way to test with the new API1.1? I predict a raft of people who have implemented into their site who have no idea about the v1.0 switch off (Tuesday 5th?), will suddenly inundate you with questions :( (like this one ;)

[purcell]

@petenorris To be clear, tweet.js will not work with the Twitter 1.1 API, because that API requires the use of authentication, and nobody with any sense will put his API authentication key inside a web page so that a widget can use it. So testing with the new API is not a concern.

However, Twitter has its own embeddable widgets, which do not require authentication keys, so it may be possible to reverse-engineer those and make tweet.js mimic the requests they make. This requires a chunk of work which I have not undertaken (so far) because it has been neither urgent nor funded. Note that tweet.js may still fall foul of the user interface guidelines, and therefore not be permitted to use the API.

It's extremely unfortunate that Twitter has declared war on API clients like this.

[petenorris]

Thanks. I really do appreciate your reply and the work to build this in the first place. My client is rather large and I couldn't use the native twitter widget as it's got a minimum height of 300px (and didn't fit the design!). I have applied stanScates (thanks Stan) to our test version of the site, and its working fine on the test site - fingers crossed for Tuesday!

[purcell]

I am also considering building a kind of "proxy" site, which would allow you to enter your Twitter auth details, and then it would sit between tweet.js and the Twitter 1.1 API, similarly to Stan's solution. If there's enough interest then I'll build such a thing, probably payable on a cheap monthly subscription basis.

[asirota]

Hi @purcell - thanks for a terrific jQuery plugin. We've used it on several websites now (www.cloudfingr.ca and www.newpathconsulting.com) and find it indispensable. We'd be happy to pay a monthly subscription cost to support your efforts since this is a feature for certain clients who wish to integrate Twitter. We have only 1 doing this so far, but maybe as Twitter takes off there will be more. The new v1.1 API must have a decent client, jQuery friendly solution that is not tied directly to the Twitter, non customizable v1.1 compatible embeddable widgets.

[raideus]

I second @asirota 's comment. At the agency I work for we have several large clients who would definitely be willing to pay an recurring fee for this type of functionality given the new restrictions.

[ghost]

I wish I could say the same as asirota and bootsz, but it's pulling teeth to get funding for anything from my employers. Even small recurring fees. If you had a free version with branding and link back to your "paid" version, I would be free to use that. I definitely appreciate the work you've put into it and feel that you should get compensated though.

[willisweb]

I'm not sure about a reoccurring fee but we would happily pay a reasonable amount for each website we use it on.

[purcell]

Thanks guys. There's already been some discussion about options. If we put in work to provide a middleman service, we're in danger of being shut off by Twitter for bypassing the Twitter TOS (and helping you guys do it), so we need to think carefully about how to proceed.

(Aside: I've just been playing with the embedded timelines, and apart from the lack of customization supported, it's ridiculous that you have to explicitly create them via the twitter site.)

[johnpscott]

I've been playing with solutions to the authentication issue. I have a work around to minimise the changes to the client side javascript. Although the official twitter stance is that they allow use of access_tokens in client side javascript, the real issue (other than security) is the handling of access_token revocation and renewal.

There seems to be a logical split between the requirement to authenticate and the tweet functionality of the plugin: the plugin is UI centric and authentication is usually a server side issue. To this end I have come up with a drop-in HTTP Handler which sits in the root of your site (ASP.NET site only for now) and serves as a proxy for twitter requests. The only change to your websites Javascript is to add a configuration entry for: twitter_api_url in the format: domain_name/twitterproxy.ashx/api.twitter.com (there may be a slight mod to the API signature but this still works for now).

The handler receives the request, retrieves an access token from configuration entries in the site's web.config file, adds the authentication header and the calls the twitter api, returning the results to your page.

There are a couple of changes necessary to jquery.tweet.js to fix changes to the format of the direct search call as the url signature has changed.

The only issue I still need to address in my handler is caching of the access_token and automatic renewal at regular intervals (twitter recommend every 15 minutes), however, this is fairly straightforward to code up.

I have this approach working with jquery.tweet.js in umbraco sites (v4.x & v6) and an MVC 4 site. The core authentication code has also been tested in various ASP.NET sites. This approach should allow a site to be update by copying one file, updating jquery.tweet.js, and adding one configuration entry to your jquery initialisation in your web page/template.

[purcell]

Thanks John. So that's similar to an ASP.NET version of Stan's PHP proxy above. If we can settle on a common request URL scheme, we can get jquery.tweet.js to work with either of those, and then maybe bundle them into the core tweet distribution, with appropriate documentation.

Note, of course, that unscrupulous developers could look at the source of your site in order to determine the proxy endpoint URL, and then configure their own widgets (or scripts!) to access that directly without authentication, which might impact the authenticated account you use.

[johnpscott]

Hi Steve, My approach doesn’t require a central proxy, it is tied to the site it is embedded in and can be disallowed from external sites (the default on IIS is not to allow direct calls to ashx handlers), even though the url is implicitly available in the jQuery startup code for the plugin.

I think that other platforms would have their own issues around this area- but purely configuration (htaccess etc).

A centralised proxy would be difficult to manage as you would have to manage consumer key/secret pairs for all the sites, or risk exceeding the rate limiting imposed by twitter by using a centralised key/secret pair.

It’s a shame that your plugin has been compromised by twitter’s draconian approach. And the double whammy of having to consider server side code.

Thanks for your hard work and great plugin.

Cheers Scotty

[purcell]

@johnpscott Agree re. the centralised proxy. It's a challenge.

Regarding disallowing calls from IIS, programs (though not javascript) could still arbitrarily make requests to the proxy endpoint with spoofed referrer headers. But that's not necessarily a big risk.

This situation is indeed all very unfortunate.

I run a site which has profile pages for hundreds of people who have twitter accounts, and to use Twitter's embedded timelines I'd have to manually create a widget for each of those twitter accounts... so even if I was happy with using their widgets, it's impractical. For that reason, I'll probably have to build something along the lines of what you and Stan have done. Sigh...

[johnpscott]

I now have a working solution for my sites with minimal impact on the original tweet.js. There is an option property called twitter_oauth_proxy and in addition modifications to the api signatures to v1.1 calls to call a local handler with the actual api call tagged on. The handler can be platform specific as it is run server side, so there is a possibility of having generic handlers (php, asp.net, ruby etc.) as well as allowing developers to integrate independently without having to modify the tweet.js (v1.1 api) codebase.

e.g. $(".hometweets").tweet({ join_text: "auto", username: "johnpscott", twitter_oauth_proxy: "/twitterproxy.ashx", count: 1, loading_text: "" });

My ASP.NET handler (twitterproxy.ashx) is also independent of the twitter client or server side code. So it can be used for any site with twitter functionality. The handler works with the web.config and a custom config file to hold twitter api keys and tokens as well as token caching parameters.

All that is required is you change http(s)://api.twitter.com/ calls to /handler.ashx/api.twitter.com/ calls. Security is maintained by only returning results to the local context and the config files are not accessible directly.

Does this look like a way forward?

[stoyicker]

Has this 'problem' been automatically solved or something? I'm running Tweet! in some tests in localhost and it works fine, when the API was supposed to have been closed to public access on March the 5th, and that's 13 days ago...

[StanScates]

@Stoyicker They were doing blackout testing on the 5th, only shutting it down for a few hours to see what impact it could have. It's not been shut down completely yet though, the next blackout test is scheduled for the 20th. Follow http://twitter.com/twitterapi for more information regarding the retirement plan :)

[squibell]

@johnpscott This is exactly what I need. Would you consider sharing the twitterproxy.ashx code ?

[johnpscott]

Ok then, had a busy week at work so only just got round to looking at this again. I've got a second pass of my proxy sorted now and it's running on http://tweet.johnpscott.co.uk (just a modified version of the tweet.js distribution example). It needed a few modifications to the tweet.js file to address changed twitter v1.1 API signatures and responses, as well as adding an entry for a proxy handler.

I'm not really conversant with github, so not sure how to put a forked version on the system, but I'm happy to do so if you're interested.

I'm not convinced the search query examples are 100% correct, so any advice is welcome. The tweet.js file can be downloaded from: http://tweet.johnpscott.co.uk/jquery.tweet.js if you want to take a look.

[purcell]

@johnpscott If you want to email me the modified code + aspx (perhaps as a zip or similar), I can commit it to a separate branch in this repo so people can try it out. My address is steve at sanityinc dot com.

[joebirkin]

@StanScates I'm having trouble using your adaptation for an array of usernames, has that functionality been removed with your version? Otherwise seems solid!

[StanScates]

@hodephinitely I wasn't aware you could pass an array of usernames, can you open an issue with the code you're using and I will integrate support for it ASAP. Thanks

[willisweb]

I was just wondering what the latest is on this. Has the @johnpscott solution been decided as the way forward? If so I'm not sure what I should be doing. Is his code going to be merged or a branch set up for us to try? Do I just download the tweet.js file from: http://tweet.johnpscott.co.uk/jquery.tweet.js and replace the old one? Are there some config changes or other things I need to set?

[purcell]

John's solution won't be for everyone, because it requires that you set up your own ASP.NET backend for the js using his code, which he hasn't yet shared.

In the meantime, Stan's PHP backend plus modified javascript (see comments above) seems to work for some people.

Ideally we'll gather into this repository the code for various platform-specific backends together with a single modified javascript file, but it's not clear yet when that will happen or who will lead the effort.

-Steve

[willisweb]

BTW I just read on https://dev.twitter.com/blog/api-v1-retirement-final-dates that the Twitter REST API v1 will officially retire on Tuesday, May 7, 2013.

They will hold another blackout test on April 16, 2013 beginning at 23:00 UTC (4:00pm PDT).

Looks like I'll have to give @StanScates method a try.

[stoyicker]

Anybody already using Stan's PHP workaround can share URL? It used to work for me but I had to format the PC where I kept it and now I think I have a problem with the modpath or something but I can't get to set it up again (stuck on 'loading tweets'), so frustrating!

[purcell]

@Stoyicker The link is in the discussion above.

[stoyicker]

@purcell If you mean http://www.cloudfingr.ca and http://www.newpathconsulting.com, I've downloaded both webpages and checked their jquery.tweet.js files, and they're using the original ones, not Stan's mod. In fact they're not even using the most updated version.

[StanScates]

@Stoyicker https://github.com/StanScates/Tweet.js-Mod

[stoyicker]

@StanScates Ok looks like I didn't explain myself properly. I know your solution. In fact you already helped me deploying it some weeks ago. What I'm looking for now is somebody who has a webpage with it working to make a comparison between both that one and mine because in mine I've tried everything I've been able to think of as modpath, as well as simply applying your solution over the default http://tweet.seaofclouds.com webpage in the root of my web (so modpath theoretically doesn't have to be specified, since it already points to /twitter/), but I keep having the infinite 'loading tweets' message, due to (Chrome says) an error in jQuery, 404 on http://localhost/twitter/, which does exist. I've searched for similar issues too but they seem to be relative to rate-limit, which is not the case since the original http://tweet.seaofclouds.com works for me. Should I open a new issue?