issue with long ids

[CarlosJimenez]

Twitter expanded user ids to bignums that overflows integers in javascript.

If I use your example with an account with a few followers with big IDs:

//  get the list of user id's that follow @johanavq
T.get('followers/ids', { screen_name: 'johanavq' },  function (err, data, response) {
  console.log(data)
})

returns

{ ids: [ 14645160, 723389637071249400, 330247364, 145781550, 52802375, 718810552609480700, 142310748, 82530518, 705167622904680400, 1194192355, 131352575, ....

3 data with bad IDS = 723389637071249400, 718810552609480700, 705167622904680400

If I dump the body received by Twitter you can see that the right ones are:

723389637071249410 718810552609480705 705167622904680448

body {"ids":[14645160,723389637071249410,330247364,145781550,52802375,718810552609480705,142310748,82530518,705167622904680448,1194192355,131352575,...

To see the original body sent by twitter i included the line

  console.log('body',body);

in twitter.js line 301

The problem comes in line 302 with the native implementation of JSON.parse that dismiss IDs accuracy

  try {
        body = JSON.parse(body)      <-  this creates the issue
  } catch (jsonDecodeError) {

I think that you should return ids as strings (as Twitter does id_str) instead to return ints.

The problem is that Javascript only uses 56 bits instead of 64 bits integers, so bignums are rounded. Twitter in fact, give id and id_str in some of the structures to address the issue in languages that don't support 64 bits integers like javascript.

Best regards Carlos

[CarlosJimenez]

More info in Twitter https://dev.twitter.com/overview/api/twitter-ids-json-and-snowflake https://groups.google.com/forum/#!topic/twitter-development-talk/ahbvo3VTIYI

Maybe this could help http://stackoverflow.com/questions/18755125/node-js-is-there-any-proper-way-to-parse-json-with-large-numbers-long-bigint https://github.com/douglascrockford/JSON-js https://www.npmjs.com/package/json-bigint

I am working in this possible workaround

var JSONbig = require('json-bigint');
...
  var onRequestComplete = function () {
    if (body !== '') {
      console.log('body',body);
/*
      try {
        body = JSON.parse(body)
        console.log('body',body);
      } catch (jsonDecodeError) {
        console.log('jsonDecodeError catched');
        // there was no transport-level error, but a JSON object could not be decoded from the request body
        // surface this to the caller
        var err = helpers.makeTwitError('JSON decode error: Twitter HTTP response body was not valid JSON')
        err.statusCode = response ? response.statusCode: null;
        err.allErrors.concat({error: jsonDecodeError.toString()})
        callback(err, body, response);
        return
      }
    }
*/
      body = JSONbig.parse(body);
      console.log('body',body);
    }

returns

{ ids: [ 14645160, { [String: '723389637071249410'] s: 1, e: 17, c: [Object] }, 330247364, 145781550, 52802375, { [String: '718810552609480705'] s: 1, e: 17, c: [Object] }, 142310748, 82530518, { [String: '705167622904680448'] s: 1, e: 17, c: [Object] }, 1194192355,

[anvaka]

I've got bitten by this today. Wrongfully assumed that 404 errors are bots on twitter... It would be nice to parse ids as strings, not numbers. And warn users that the data returned by the library isn't accurate.

[anvaka]

@CarlosJimenez one way to workaround this for your particular case is to use stringify_ids in the query string. I assume this should work:

T.get('followers/ids', { screen_name: 'johanavq', stringify_ids: true },  function (err, data, response) {
  console.log(data)
})

[CarlosJimenez]

Thank you very much. Works like a charm!!!