Plan for upcoming Twitter API deprecation

[knolleary]

Twitter have now confirmed a date for when some of their deprecated APIs will be turned off. We need to get the Twitter node updated as soon as possible to minimise the disruption this will bring.

Most of the functionality is still available, but not all...

Twitter In

• the public stream for tweets containing the configured search term
  • uses https://stream.twitter.com/1.1/status/filter
  • no change needed
• all the tweets from accounts that the authenticated user follows
  • uses deprecated https://userstream.twitter.com/1.1/user.json
  • migrate to https://api.twitter.com/1.1/statuses/home_timeline.json - polled once a minute
• all tweets by specified users
  • https://api.twitter.com/1.1/statuses/user_timeline.json
  • no change needed
• direct messages received by the authenticated user
  • uses deprecated https://api.twitter.com/1.1/direct_messages.json
  • migrate to https://api.twitter.com/1.1/direct_messages/events/list.json - polled once a minute
• twitter events for the authenticated user
  • uses deprecated https://userstream.twitter.com/1.1/user.json
  • No viable alternative. The AccountActivity API relies on WebHooks - which we cannot use in non-public-cloud environments.

Twitter Out

• post status
  • uses https://api.twitter.com/1.1/statuses/update.json
  • no change needed
• post status with image attached
  • uses deprecated https://api.twitter.com/1.1/statuses/update_with_media.json
  • Follow migration guide - upload media first then use /statuses/update to tweet
• post new direct message
  • uses deprecated https://api.twitter.com/1.1/direct_messages/new
  • Migrate to https://api.twitter.com/1.1/direct_messages/events/new.json

[knolleary]

@andypiper I think I have covered everything above. Are you aware of an up to date node.js client library? We use twitter-ng which is riddled with deprecated APIs and don't really want to take on the task of updating it.

[ukmoose]

Is the node also impacted by the requirement to whitelist callback urls as part of the OAuth step? This change comes in in 20 days.

https://twittercommunity.com/t/sign-in-with-twitter-users-must-whitelist-callback-urls/105342

[knolleary]

Sigh. Yes. That breaks us completely.

[andypiper]

Unfortunately the callback URL thing is critically important (to implement on the Twitter sidee, and in terms of the impact on the developer side). It means each instance of NR would need its own API key and secret - which to be honest is a better way of doing things, as it means that they are not all competing for rate limit on connections. It is a much greater setup overhead though, which is a shame.

Not currently aware of better Node implementations, our Account Activity Dashboard sample is doing things directly in Node rather than relying on specific library wrappers.

Why would the AA API webhooks be a blocker - NR can present an additional flow or set of flows that provide the callback endpoints, potentially? I'll have a play with this.

[knolleary]

Just to record progress, got the Twitter In node updated locally to use the apis described above. One side effect is you hit the rate limit really easily if you deploy more than 15 times in 15 minutes - something the old streaming api was more tolerant of. May need to turn-down the polling frequency to make it slightly more tolerant.

We cannot use the AccountActivity api for the simple fact that many of our users run on home networks where they cannot expose the necessary webhooks on the public internet safely.

It would be less of an issue with cloud-hosted instances - but we simply don't have the resources to maintain multiple versions of the node. The fact I'm only now beginning to look at the changes needed without only about 20 days to go before the old apis are turned off reflects the fact that no-one else has offered to help on it. I hope to get the node updated by the deadline, but equally, I have a 10 day holiday between now and then.

[Yukibashiri]

Any news?

[knolleary]

News about what? The Twitter node has been updated inline with the comments above. Your comment has reminded me this can be closed.

[knolleary]

Ah right. This issue was primarily focused on the API changes that were introduced - all of which were fixed in the Twitter node. I forgot I had also used this issue to track updates to support 280 chars - something that hasn't been done. The main issue is the library we use under the covers doesn't support it and is not actively maintained. So until someone reimplements the streaming support we use from that library, not much is going to happen here.

I'll reopen so to stays on the todo list.