pagination?

[evantahler]

It doesn't look like this library handles pagination for large result sets. Do you have any advise?

[michaelcarter]

Hi @evantahler, sorry for the late reply.

I don't think Mixpanel's API actually offers a way to limit the amount of data returned per request, except for limiting the number of days data you request from it. Obviously if your site is very busy then this could still be a rather large amount of data.

Is your particular issue that you'd like to paginate through results once you've actually received them or that you'd like to be able to request less from the API for faster response times?

[evantahler]

I ended up implementing this in user-space for the "engage" api, which looks like it returns data in blocks of 1000 users. The good news is that the JSON you get back from Mixpanel contains the session and page, so it's easy to do:

connection.prototype.getAll = function(method, where, dataCallback, doneCallback, sessionId, page){
  var self = this;
  if(page === undefined){ page = 0; }

  var payload = {};
  if(where){     payload.where = where;          }
  if(page){      payload.page = page;            }
  if(sessionId){ payload.session_id = sessionId; }

  panel[method](payload, function(error, data){

    // who changes the order of response params?!
    if(data === undefined && error.status === 'ok'){
      data = error; error = null;
    }

    if(error){
      doneCallback(error);
    }else{
      sessionId = data.session_id;
      var results = [];
      data.results.forEach(function(elem){
        results.push( elem.$properties );
      });

      if(results.length > 0){
        dataCallback(null, results, function(){
          self.getAll(method, where, dataCallback, doneCallback, sessionId, (page + 1));
        });
      }else{
        doneCallback();
      }
    }

  });
};