ttezel / twit

Twitter API Client for node (REST & Streaming API)
4.31k stars 568 forks source link

GET followers/ids partially returns invalid ids #427

Open stronghandsdeeppockets opened 6 years ago

stronghandsdeeppockets commented 6 years ago

First of all, excuse my rough language, programming isn't quite my strongest point.

I managed, with some external help, to create a js script that fetches the followers of a certain user. However, some users' ids are completely wrong (and by "some", I mean almost half).

A Twitter id must have 8,9 or 10 characters, but some of mine have almost double:

959854874354532400
364099683
875588591233024000
875912710214492200
910852772454580200
943125935104049200
756016512268394500
27391643
317688032

You may notice that the invalid ids all have 18 chars and they always end up "00". All the other ids are accurate.

This is the code:

var Twit = require('twit')
var fs = require('fs')

var output = ''
var cursor = -1
var id_page = ''

var T = new Twit({
  consumer_key:         'yes',
  consumer_secret:      'nope',
  access_token:         'yess',
  access_token_secret:  'nopep',
  timeout_ms:           60*1000,  // optional HTTP request timeout to apply to all requests.
})

function get_them_ids(){
    if(cursor != 0){
        T.get('followers/ids', { screen_name: 'aisaifu', cursor: cursor },  function (err, data, response) {
            id_page = data['ids'].join('\n')
            output += id_page
            cursor = data['next_cursor']
            get_them_ids()
        })
    }
    else{
        fs.writeFile('id_list.txt', output, 'utf8')
    }
}

get_them_ids()

The code breaks the 5,000 barrier imposed (something about cursors) and it manages to pull 30k ids. The code also is supposed to put each id on a separate line. I took a sample of 1,000 ids from the 30,000 and tested how many were valid. Only 208 are.

Both me and my friend are clueless about what it could cause this issue.

diego1199 commented 6 years ago

I'm pretty sure this is the problem https://developer.twitter.com/en/docs/basics/twitter-ids

JavaScript cant handle properly the ids because the unsigned 64-bit integers and the ids are becoming invalid at the parser of the library. The library must return the ids on strings instead of numbers.

diego1199 commented 6 years ago

I found the solution and now all the ids i'm receiving are working. You must add the parameter: stringify_ids : true to the call.

Like this: T.get('followers/ids', { screen_name: 'aisaifu', cursor: cursor, stringify_ids : true }, function (err, data, response) { ...

@stronghandsdeeppockets

stronghandsdeeppockets commented 6 years ago

@diego1199 Thank you very much for the fix. Can you recommend a mass id converter? Or do I have to use the API again?

diego1199 commented 6 years ago

You may want to use the users/lookup of the API. That method allows you to send a request of up to 100 ids or screen_names and it will return the full user objects.

dandv commented 6 years ago

@stronghandsdeeppockets: what would you like to convert?

BTW we've documented this gotcha with IDs in our twitter-lite library.